illumos gate

If an incomplete character is passed to mbtowc(3), it fails to set errno to EILSEQ, even though that is unambiguously required by POSIX.

Setting errno in that case makes a lot of sense because having a function that can fail and that sets errno in some failure modes but does not set errno in other failure modes would be a terrible idea. Such a function would be ridiculously complicated to use. To detect the reason for failure, you would have to:

- save errno
 - reset errno to zero
 - call the function
 - inspect the return value to detect failure
 - inspect errno to decide about the reason for failure
 - if errno is zero, restore the saved errno

That is completely unreasonable, in particular for a seemingly innocous function like mbtowc(3). Next to no programmer would get that right in any real-world program.

Note that this bug is very widespread, it also affects FreeBSD and glibc, and also OpenBSD until i fixed it today. I will also send a note around to the other systems.

The OpenBSD patch is here:
http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/lib/libc/locale/mbtowc.c

The attachment contains a patch against illumos-gate -current, mainly to make it even clearer what exactly i mean. Note that i did not test the patch since i don't have an illumos system - so please test before commit! Thanks.