Project

General

Profile

Feature #12358

Need mbrtowc variant that indicates consumed zero bytes

Added by Robert Mustacchi 9 months ago. Updated 8 months ago.

Status:
Closed
Priority:
Normal
Category:
lib - userland libraries
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

To properly implement support for open_wmemstream() we need to be able to write embedded zero byte wide characters. Unfortunately, the interfaces that we have today have no way of indicating how many bytes was actually consumed when the zero is translated. This makes it very difficult to correctly write a program here as there is no guarantee a locale uses a single byte and in fact looking through many of the locales, it is not always the case that there is a single byte. To rectify this, I'd like to add a private function to libc which does a variant of mbrtowc() which understands this. We could consider making this public, but to do so we should work with the broader open source communities to agree on a name for this.

To summarize, this allows a variant of mbrtowc() to always indicate the number of bytes consumed, even if it results in a wide-zero which would normally only indicate 'zero'.


Related issues

Related to illumos gate - Feature #7092: Want support for stdio memory streamsClosedRobert Mustacchi

Actions
#1

Updated by Robert Mustacchi 8 months ago

To test this, I imported the OpenBSD mbrtowc regression tests into the stdio test suite as a part of (7092). open_wmemstream(3C) uses this extensively internally, giving me enhanced confidence of the change. I also have been using the en_US.UTF-8 locale while doing work on a system with these changes.

#2

Updated by Robert Mustacchi 8 months ago

  • Related to Feature #7092: Want support for stdio memory streams added
#3

Updated by Electric Monk 8 months ago

  • Status changed from New to Closed
  • % Done changed from 90 to 100

git commit 0ac311bae7f6f50d9ba506b52bd8860f2d68d4ce

commit  0ac311bae7f6f50d9ba506b52bd8860f2d68d4ce
Author: Robert Mustacchi <rm@fingolfin.org>
Date:   2020-03-26T07:42:53.000Z

    12358 Need mbrtowc variant that indicates consumed zero bytes
    Reviewed by: John Levon <john.levon@joyent.com>
    Reviewed by: Yuri Pankov <ypankov@fastmail.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

Also available in: Atom PDF