illumos watch for June

Debugging gcc built binaries with mdb

I became curious if debugging with mdb was still possible for dumps and binaries produced by gcc compiler suite. I got my answer on #illumos IRC channel:

11:50 < madwizard> A stupid question maybe, but IIRC, I can't debug with (k)mdb dumps from gcc/clang produced binaries, am I right?
11:54 < jperkin> um?
12:00 < jperkin> ('no' is perhaps a more useful answer, just curious why you'd include gcc in the question)
12:05 < madwizard> jperkin: For distro built with gcc. Just curiosity, if that means switching to gdb
12:06 < jperkin> ok, well a stupid question of my own, are there any distros _not_ built with gcc? I assumed studio was long long gone
12:06 < madwizard> Commercial distro
12:07 < madwizard> jperkin: So to answer my quetsion - no
12:07 < madwizard> I need to debug with gdb
12:07 < jperkin> anyway, ABI is the important factor rather than compiler, so yes you can continue to use mdb, though the compiler can affect whether you can get anything useful (just ensure you
                 don't use -fomit-frame-pointer, we disable it in SmartOS gcc to force useful dumps)
12:07 < madwizard> oh
12:08 < madwizard> So I am mistaken
12:08 < jperkin> the question was if you were right ;)
A lesson to remember, also, understand your own question before parsing the answer. :)

illumos alternate bootloader now has it's own wiki page

Toomas Soome posted a link to the wiki page that hosts alternate boot loader project. Personally I'm waiting for results with lots of interest.

Gordon Ross sent a mail with more details about user-mode smb server, following his mail from last month.

He explains that it's a work that was done to ease development od smb server code and speed up build. In his own words from the mail:
So what's this "fksmbd" thing, etc? I think this is really cool...

As I described at illumos day last year [1] we found it incredibly
helpful to be able to develop and debug the SMB server kernel module
as part of a user space process. This (along with the "quick" build
scripts) allows us to enjoy edit/compile/debug cycles of under a
minute, and full source-level debugging when we want it, as well as
full mdb and dtrace support. (Yeah, wow!)
Read the mail for full explanation. Links to Gordon's talk below:

Matt Ahrens asked for review of new cryptographic checksums for ZFS: SHA-512, Skein, Edon-R

As explained in the review board entry:
4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R
Note that this allocates new checksum algorithm values. If you have developed new checksum algorithms on a private branch, now would be a good time to speak up.

enum zio_checksum {
    ZIO_CHECKSUM_NOPARITY, // == 10, last one in illumos
    ZIO_CHECKSUM_SHA512,   // == 11, proposed addition
    ZIO_CHECKSUM_SKEIN,    // == 12, proposed addition
    ZIO_CHECKSUM_EDONR,    // == 13, proposed addition
Note that this doesn't change any defaults, e.g. "zfs set dedup=on" still changes the checksum algorithm to sha256. You need to explicitly opt in to use the new checksum algorithms. We'll probably eventually change that default for dedup=on to sha512. (This is the same strategy we used with checksum=on and lz4.)

Sergio Aguayo asked question that may be interesting to any person struggling with swap usage during illumos build

Hello guys

I'm having some problems with very high swap usage during linted builds.

Currently the VM I'm using has 6GB RAM and 1.5GB swap. The problem is that it runs out of swap, while it still has tons of free ram (about 4gb of free ram). Another interesting issue is that the swap usage doesn't decrease when the build finishes (or, more precisely, fails). Swap usage only drops down to near zero after the build finishes successfully (which doesn't happen very often). Any ideas on what can be causing this?


Sergio Aguayo

Some discussion followed, but Richard Lowe seems to have found the reason:
The usual cause for this is lint. lint dumps a couple of _very large_ files under /tmp by default, setting TMPDIR should fix that if you need to. It's perhaps also worth noting that if lint crashes out of memory _because_ of these temporary files, it will fail to clean them up, and you'll spiral. Soon Sergio replied:
Hello guys

Thanks all for your help. Changing TMPDIR made the trick!


Alexander Pyhalov sent HEADS UP message to OpenIndiana Development mailing list:


I'm glad to inform you that I've just updated IPS to version close to
2015 Oracle IPS, based on work which Igor Kozhukhov has done for
EveryCity ( ).

I've checked that all basic operations work. Also checked work with
zones. Now our IPS supports linked images. The most interesting for me
in this update is migration to Python 2.7.
Basic operations work with both python versions, but test suite
discovers that there are some incompatibilities with Python2.6. I
suppose in everyday usage you'll hardly notice this, but if you do, set
python mediator to 2.7 version. In any case we are going to move to
Python 2.7 as our main system Python soon.
He later goes on about tests that were performed. Read the link above to see whole message.

Aurélien Larcher sent mail about Firefox installed from mozilla/contrib crashing repeatedly on Save As operation.

He is not the first one to experience problems with FireFox and Thunderbird from Mozilla contrib crashing. There's been lots of discussion about it on #oi-dev and #openindiana channels.
As Alexander Pyhalov explained in his mail:
I've just committed fix for pango (it found libthai on build server,
built and installed, but didn't deliver it). But I
think this is not your main problem.

I've disabled building of this module, because I don't think everyone is
happy to have libthai installed.

In related note but on OpenIndiana Discuss mailing list Jonathan Adams explaining why he abandons OI for his desktop, quoting Firefox and Thunderbird crashing as major contributions.

Peter Tribble on the other hand Tribblix

After several previous attempts ended in failure, I've recently managed
to get a functional build of LibreOffice on my own Tribblix distribution.

The binaries I've created probably aren't useful elsewhere (not so much
a problem with the binaries themselves as all the missing prerequisites)
but as a pointer for people who want to get LibreOffice running on other
distros such as OI, I've written up my notes here:
Visit the link above for technical discussion of build process and requirements.

Jean-Pierre André announced new version of ntfs-3g and fuse kernel modules

A new version of ntfs-3g and the fuse kernel module is available on

The fuse kernel module can now cache the file attributes for not
hard-linked files, which brings its performance on par with other
OSes. Below are the measured times for compiling xz for Windows
on a triple-boot desktop. All the needed files (source and header
files) are stored in a shared NTFS partition, and the same
compiler is used. There were 90 source files to compile, each of
them including 20 to 30 header files.
The package was compiled twice on each OS, showing a difference
related to data page cacheing.

                             real       user        sys
Linux Fedora 21           0m10.811s   0m6.835s   0m2.374s
         same repeated      0m7.697s   0m6.448s   0m2.149s
Windows 10                0m11.484s
         same repeated      0m6.078s
OpenIndiana Hipster 2015   0m9.754s   0m5.962s   0m2.792s
         same repeated      0m7.660s   0m5.957s   0m2.758s

There are few commits regarding SMB that I know at least few people were waiting for.

Access Based Enumeration not working after 1527 - this was regression introduced by previous commit.

SMB server fails setting dates earlier than 1970. Obviously best solution would be to find a way to store date set by user/application, but throwing an error is

smbd workers documentation

Gordon also took on the task of documenting smbd workers taskq kstats. His attached draft got comments from Garrett D'Amore. Hopefully this will lead to better documentation. kstats are very useful tool and many people would welcome clean docs.

Multiple SMB fixes from Gordon Ross

Gordon Ross have sent an e-mail pointing to few SMB issues: More SMB server reviews: 6020, 6021, 6022, 6023, 6024, 6025, 6026 asking for reviews.

6020 panic in smbsrv netbios_first_level_name_decode

6021 Windows Explorer Stops Working / Crashes when Security Tab is accessed

6022 MacOS logs error messages when trying to lookup users

6023 Quota tab for an SMB share sometimes doesn't work

6024 ACL editor crash in Windows 2012 R2

6025 Customer cannot set Share Properties using MMC

6026 Zero-padded IP address strings returned by SMB

That immidiately followed with LGTM (Looks Good To Me) from Igor Kozhukhov.

Again from Dan McDonald - Say hello to very-larval i40e

Dan posted a mail (link above) about starting work on i40e driver. In his own words:
Just so y'all know, this is the very earliest intro to the i40e driver (Intel 40Gig Ethernet, aka. Fortville, aka. the XL710 board). There are LOTS of KEBE comments (my work-in-progress markers) with more to come, there are TONS of missing pieces of functionality, and I haven't even tried attaching it to a real piece of hardware yet.

I *do* have real hardware, however, thanks to Intel - a pair of XL710s that I will connect back-to-back for testing.

NVMe driver from Hans Rosenfeld

Hans Rosenfeld announced a NVMe driver (Specs here) which, as I understand, is targeted at better handling Solid State Drives. There's been a review from Robert Mustacchi and an answer from Hans so far. So if you'd like to see that driver in illumos and are capable of doing a code review, why not join in?

Remote stale locks detection

Marcel Telka opened an issue that seems very interesting. Support for remote stale lock detection adds capability to detect stale locks in NLM, NFSv4 and SMB. As described by Marcel, there's a possible scenario when client releases a lock, but server got out of sync and never notices. Other client may not be able to obtain the lock then. Marcel wants to implement ability for checking for remote stale locks. For better explanation visit the issue report by Marcel.

Joshua M. Clulow posted a request

In a mail titled ACPI MCFG Tables and You Joshua asked for a some data from running illumos systems:

Hi folks,

(tl;dr) If you have a running illumos system, and a few moments, I would very much appreciate your help in gathering data from as many different systems as possible. This mail contains a detailed problem description, but you can also just skip to the "GATHERING INFORMATION" section below.

You can see if your system provides an MCFG table by checking if fakebop was able to insert the "ecfg" system property:

prtconf -v /devices | awk 'q{print;exit}/ecfg/{print;q=1}'

You should get either two lines of prtconf output, or nothing. I would also like to know what address your illumos system believes to be the MMIO config base address:

mdb -ke "mcfg_mem_base/J" | tail -1

Just so you don't need to dig it up from the mail thread, there's been at least one person having trouble running command from Joshua:

Hmmm... I guess the awk command was damaged in transit. I just get awk errors (but I hate awk and never use it so it serves me right .

Do you meam something like this?

# uname -a
SunOS nfs 5.11 omnios-170cea2 i86pc i386 i86pc

# devprop -vi ecfg

Reason for problem was found pretty soon:
Perhaps it's a nawk vs awk thing. I seem to recall we replaced awk with nawk in SmartOS a while ago.

No closed/opened issues table this month

I have been pointed towards few errors in my tables in previous posts. Until I'm able to produce proper reports, I won't be posting it here.