Project

General

Profile

Bug #10167

unable to check out illumos-gate on Windows

Added by Garrett D'Amore 7 months ago. Updated 3 days ago.

Status:
Closed
Priority:
Normal
Category:
cmd - userland programs
Start date:
2018-12-28
Due date:
% Done:

100%

Estimated time:
Difficulty:
Bite-size
Tags:
needs-triage

Description

I recently decided to try working on Windows for development, and I checked out a copy of the illumos-gate repo on Windows using the Microsoft supplied copy of git that ships with Visual Studio.

The checkout failed, due to occurrences of file names that are "illegal" for Windows -- specifically usr/src/cmd/mailx/aux.c and usr/src/cmd/sendmail/aux/ -- these refer to the an "AUX" device and so these filenames are not legal on a Windows system (even with the extension.)

I have a weird feeling that I might be the first person to try this. (We're also exploring cross compilation of the gate from Windows, but that's another matter.)

We should rename the aux/ directory for sendmail, and the aux.c file in mailx.

History

#1

Updated by Igor Kozhukhov 7 months ago

move sendmail + mail/mailx outside illumos-gate.
because they are both are userland apps.
for example: on DilOS we are using exim as sendmail replacement , and bsd-mailx.

#2

Updated by Joshua M. Clulow 7 months ago

If we're going to accommodate clones on Windows, we should probably add to pbchk with the full list of forbidden file names.

#3

Updated by Toomas Soome 7 months ago

Igor Kozhukhov wrote:

move sendmail + mail/mailx outside illumos-gate.
because they are both are userland apps.
for example: on DilOS we are using exim as sendmail replacement , and bsd-mailx.

no. at least 90% of illumos issues are rooted to the fact that illumos-gate is not operating system but stripped down skeleton. illumos-userland is not touched for 7 years. we need the functional operating system, not some sort of dissected dead body.

#4

Updated by Gergő Mihály Doma 7 months ago

The checkout failed, due to occurrences of file names that are "illegal" for Windows -- specifically usr/src/cmd/mailx/aux.c and usr/src/cmd/sendmail/aux/ -- these refer to the an "AUX" device and so these filenames are not legal on a Windows system (even with the extension.)

I'm using the Cygwin version of Git on Windows for this reason.

Note:
It's possible to use these "illegal" filenames in Windows, if you use DOS Device paths, see these command line examples:
Creating the C:\AUX directory:

MD \\?\C:\AUX

Creating an AUX.txt in C:\AUX:
FSUTIL file createNew \\?\C:\AUX\AUX.txt 0

Deleting AUX.txt:
DEL \\?\C:\AUX\AUX.txt

Deleting AUX directory:
RD \\?\C:\AUX

So, from an another viewpoint, this is an issue of insufficient handling of path-names in Git (most of Windows application suffer from this, even the Windows GUI).

We should rename the aux/ directory for sendmail, and the aux.c file in mailx.

I agree with this solution.
Much easier than to fix up every [yes, literally] application on Windows.

Sources:
Windows Dev Center: Naming Files, Paths, and Namespaces
Windows Dev Center: File path formats on Windows systems

#5

Updated by Garrett D'Amore 15 days ago

  • Assignee set to Garrett D'Amore
  • % Done changed from 0 to 80

There are two problems. One is mailx/aux.c which can be trivially renamed to util.c

The second problem is that a recent change to the test locale support (zz_AA locale) introduced a symbolic link, which doesn't work on all systems. (The solution to that is to remove the symlink, and add a Makefile rule that uses the symlink source when generating the output -- en_US.UTF-8 is the symlink source.)

#6

Updated by Rich Lowe 15 days ago

If you're really going to "fix" this, you should add the git magic to do line-ending conversion at least sufficient that nobody checks in DOS endings.

#7

Updated by Joshua M. Clulow 14 days ago

If symbolic links aren't allowed in the gate, we should also update git pbchk so that it complains about them in addition to the verboten file names.

#8

Updated by Garrett D'Amore 14 days ago

I generally agree it would be nice to add those checks to pbchk.

Note that I work regularly on Windows now, and the line-endings is (usually) a problem. Git's default operation is safe.

The exception is for DTrace files, because if you copy the file (e.g. via scp) from a Windows box to an illumos box, it won't work because DTrace can't cope with CRLF.

CRLFs generally won't be committed into the tree however. (At least with default settings.)

I'd like to fix the current problems, and follow up with pbchk settings to correct this -- mostly because I'm less familiar with the pbchk machinery, whereas the inability to check out on Windows is actually painful right now.

#9

Updated by Rich Lowe 14 days ago

I really don't mind about windows "breaking" again, so sure. What I do care about is windows breaking other things, hence worry about line endings.

I remembered last night we also care about spurious executable bits, which windows people tend to leave everywhere.

#10

Updated by Garrett D'Amore 14 days ago

As I said, git by default does sane things with line endings. It checks them out using local syntax, but leaves them in the tree with UNIX syntax. This is normally the best option.

The exception to this is are those few files that would be corrupted by CRs being injected on any platform. This is notably true for DTrace, which falls over when it sees a carriage return.

I don't know how it works, but I've yet to see Windows add execute bits to a file that I created locally. I'll do some experimentation locally. I don't think we have anything in git that is executable (other than directories) right now, so that's something we could put in pbchk.

#11

Updated by Andy Fiddaman 3 days ago

FWIW, whether it's right or not, we do have around 270 objects in the repository with execute bits.

% git ls-tree -r HEAD | grep -v 644 | head
100755 blob 18e814865a54f94fb81127fd0bf1b52e9350c530    usr/src/boot/sys/boot/common/linenoise/LICENSE
100755 blob a285410678fb0ee8773cab2eff4fa97531de9714    usr/src/boot/sys/boot/common/linenoise/Makefile
100755 blob c845673cd41cf9ad235cd2bbb66d1b0b2efea6a6    usr/src/boot/sys/boot/common/linenoise/README.markdown
100755 blob a2f0936ede4423b9664319467d4425e4df061df4    usr/src/boot/sys/boot/common/linenoise/example.c
100755 blob e3a72151a4a18740c05ec61001abb90c30593b16    usr/src/boot/sys/boot/common/linenoise/linenoise.c
100755 blob fbb01cfaad84d0662d909b02ce17f6415504a9b3    usr/src/boot/sys/boot/common/linenoise/linenoise.h
100755 blob 0cb1b16b76a2acde04301751eca31427124af4c6    usr/src/boot/sys/boot/common/newvers.sh
100755 blob 93709853dc922bccbaf8b1f70effef85e9191860    usr/src/cmd/boot/scripts/create_diskmap.ksh
100755 blob 51c63de4a871f5a132f53a3adc61007e606fc8af    usr/src/cmd/dispadmin/scheduler.xml
100755 blob b217d51623c247728d05eb356006bd5e4d7f52b3    usr/src/cmd/dispadmin/svc-scheduler
#12

Updated by Garrett D'Amore 3 days ago

The shell scripts are probably appropriate to be executable. The linenoise stuff in boot I can't explain -- I don't think Toomas (where I'm pretty sure that came from) uses Windows for development... but I could be mistaken.

At any rate I'd prefer to have someone else add a git check on that if they really care about it.

#13

Updated by Electric Monk 3 days ago

  • Status changed from New to Closed
  • % Done changed from 80 to 100

git commit 955eb5e1fb25fc64c2c34501c3d9426057e3b1f8

commit  955eb5e1fb25fc64c2c34501c3d9426057e3b1f8
Author: Garrett D'Amore <garrett@damore.org>
Date:   2019-07-18T16:41:16.000Z

    10167 unable to check out illumos-gate on Windows
    Reviewed by: Andy Fiddaman <omnios@citrus-it.co.uk>
    Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
    Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

Also available in: Atom PDF