Bug #11849
closedlisten of IPv6 address fails with EAFNOSUPPORT
100%
Description
As reported here:
https://github.com/joyent/smartos-live/issues/853
attempting to listen to an IPv6 socket gives:
so_socket(PF_INET6, SOCK_STREAM, IPPROTO_TCP, 0x00000000, SOV_DEFAULT) = 4 ioctl(4, TCGETA, 0x0803C3BE) Err#22 EINVAL llseek(4, 0, SEEK_CUR) Err#29 ESPIPE ioctl(4, TCGETA, 0x0803C3BE) Err#22 EINVAL llseek(4, 0, SEEK_CUR) Err#29 ESPIPE fcntl(4, F_SETFD, 0x00000001) = 0 fcntl(4, F_GETFL) = 2 fcntl(4, F_SETFL, FWRITE|FNONBLOCK) = 0 listen(4, 2, SOV_DEFAULT) Err#124 EAFNOSUPPORT
There is a good chance this is related to the GCC 7 switch.
Updated by John Levon over 2 years ago
Here's a simpler testcase from the perl test suite:
#!/usr/bin/perl use strict; use warnings; use Config; BEGIN { if (-d "lib" && -f "TEST") { my $reason; if ($Config{'extensions'} !~ /\bSocket\b/) { $reason = 'Socket extension unavailable'; } elsif ($Config{'extensions'} !~ /\bIO\b/) { $reason = 'IO extension unavailable'; } if ($reason) { print "1..0 # Skip: $reason\n"; exit 0; } } if ($^O eq 'MSWin32') { print "1..0 # Skip: accept() fails for IPv6 under MSWin32\n"; exit 0; } } use Test::More; plan tests => 2; use IO::Socket::INET6; my $listen = IO::Socket::INET6->new(Listen => 2, Proto => 'tcp', Timeout => 15, Blocking => 0, ) or die "$@"; # TEST is($listen->blocking(), 0, 'Non-blocking works on listeners'); my $port = $listen->sockport; if(my $pid = fork()) { # Connect to ourselves with a non-blocking socket my $sock = IO::Socket::INET6->new(PeerAddr => '::1', PeerPort => $port, Blocking => 0); # TEST is($sock->blocking(), 0, 'Non-blocking works on outbound connections'); undef($sock); } elsif (defined $pid) { my $sock = $listen->accept(); my $line = <$sock>; $listen->close; exit; } else { die $!; }
Updated by John Levon over 2 years ago
$ pfexec dtrace -c ./blocking.t -n 'syscall::listen:entry { self->t = 1; } fbt:::entry /self->t/ {} proto_verify_ip_addr:return { trace(arg1); } syscall::listen:return { self->t = 0; }' dtrace: description 'syscall::listen:entry ' matched 33794 probes 1..2 IO::Socket::INET6: listen: Address family not supported by protocol family at ./blocking.t line 34. # Looks like your test exited with 124 before it could output anything. dtrace: pid 568199 has exited CPU ID FUNCTION:NAME 4 52516 listen:entry 4 52676 getsonode:entry 4 15649 getf:entry 4 11682 set_active_fd:entry 4 11682 set_active_fd:entry 4 52229 socket_listen:entry 4 52474 so_listen:entry 4 7262 mutex_owned:entry 4 7262 mutex_owned:entry 4 47747 tcp_listen:entry 4 49353 squeue_synch_enter:entry 4 48581 tcp_do_listen:entry 4 49287 tcp_bind_check:entry 4 49577 proto_verify_ip_addr:entry 4 49578 proto_verify_ip_addr:return 124 4 48887 squeue_synch_exit:entry 4 22963 releasef:entry 4 17062 clear_active_fd:entry 4 17720 cv_broadcast:entry 4 21707 set_errno:entry
Updated by John Levon over 2 years ago
$ pfexec dtrace -c ~/blocking.t -n 'proto_verify_ip_addr:entry /pid == $target/ { print(arg0); print(args[1]->sa_family); }' dtrace: description 'proto_verify_ip_addr:entry ' matched 1 probe 1..2 IO::Socket::INET6: listen: Address family not supported by protocol family at /export/home/gk/blocking.t line 34. # Looks like your test exited with 124 before it could output anything. dtrace: pid 618701 has exited CPU ID FUNCTION:NAME 6 49576 proto_verify_ip_addr:entry int64_t 0x1a sa_family_t 0x670b
Updated by John Levon over 2 years ago
The problem is in tcp_do_listen():
4352 if (sa == NULL) { 4353 sin6_t addr; 4354 sin_t *sin; 4355 sin6_t *sin6; 4356 4357 ASSERT(IPCL_IS_NONSTR(connp)); 4358 /* Do an implicit bind: Request for a generic port. */ 4359 if (connp->conn_family == AF_INET) { 4360 len = sizeof (sin_t); 4361 sin = (sin_t *)&addr; 4362 *sin = sin_null; 4363 sin->sin_family = AF_INET; 4364 } else { 4365 ASSERT(connp->conn_family == AF_INET6); 4366 len = sizeof (sin6_t); 4367 sin6 = (sin6_t *)&addr; 4368 *sin6 = sin6_null; 4369 sin6->sin6_family = AF_INET6; 4370 } 4371 sa = (struct sockaddr *)&addr; 4372 } 4373 4374 error = tcp_bind_check(connp, sa, len, cr, 4375 bind_to_req_port_only);
at :4374, sa is a pointer to "addr": but that part of the stack is now out of scope. So we can get junk at will. As it happens, GCC7 changed things enough that this happens when the address family is IPv6.
4352 if (sa == NULL) { tcp_do_listen+0xc0: testq %r13,%r13 tcp_do_listen+0xc3: je +0x1df <tcp_do_listen+0x2a8> 4357 ASSERT(IPCL_IS_NONSTR(connp)); %r12 == connp tcp_do_listen+0x2a8: testb $0x10,0xd(%r12) tcp_do_listen+0x2ae: je +0x1c4 <tcp_do_listen+0x478> %eax = connp->conn_family tcp_do_listen+0x2b4: movzwl 0x1ca(%r12),%eax 4360 len = sizeof (sin_t); tcp_do_listen+0x2bd: movl $0x10,%r11d 4359 if (connp->conn_family == AF_INET) { tcp_do_listen+0x2c3: cmpw $0x2,%ax tcp_do_listen+0x2c7: je +0x40 <tcp_do_listen+0x309> tcp_do_listen+0x2c9: cmpw $0x1a,%ax tcp_do_listen+0x2cd: movl $0x20,%r11d tcp_do_listen+0x2d3: je +0x34 <tcp_do_listen+0x309> 4365 ASSERT(connp->conn_family == AF_INET6); tcp_do_listen+0x2d5: movl $0x10fe,%edx tcp_do_listen+0x2da: movq $0xfffffffff7bf6a4d,%rsi tcp_do_listen+0x2e1: movq $0xfffffffff7c082d8,%rdi tcp_do_listen+0x2e8: movl %r9d,-0xa0(%rbp) tcp_do_listen+0x2ef: movl %r11d,-0x9c(%rbp) tcp_do_listen+0x2f6: call +0x42a5035 <assfail> tcp_do_listen+0x2fb: movl -0xa0(%rbp),%r9d tcp_do_listen+0x302: movl -0x9c(%rbp),%r11d tcp_do_listen+0x309: leaq -0x88(%rbp),%r13 tcp_do_listen+0x310: jmp -0x24c <tcp_do_listen+0xc9> 4374 error = tcp_bind_check(connp, sa, len, cr, 4375 bind_to_req_port_only); %rsi = sa (that is, the &addr stack address) tcp_do_listen+0xc9: movq %r13,%rsi tcp_do_listen+0xcc: movl %r9d,%r8d tcp_do_listen+0xcf: movq %r14,%rcx tcp_do_listen+0xd2: movl %r11d,%edx tcp_do_listen+0xd5: movq %r12,%rdi tcp_do_listen+0xd8: call +0x231b3 <tcp_bind_check> But where is: 4361 sin = (sin_t *)&addr; 4362 *sin = sin_null; 4363 sin->sin_family = AF_INET; 4367 sin6 = (sin6_t *)&addr; 4368 *sin6 = sin6_null; 4369 sin6->sin6_family = AF_INET6;
So, it looks like what's happening is that GCC has realised that we're doing writes to stack that's about to go out of scope, so it is free to elide those writes altogether. Unlike some of its other optimization choices, this very much seems like fair game - and the only right fix is to fix the scope of "addr".
Updated by Electric Monk over 2 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100
git commit 3ef756924267e272e432add760e4dc8597faef89
commit 3ef756924267e272e432add760e4dc8597faef89 Author: John Levon <john.levon@joyent.com> Date: 2019-10-23T16:49:08.000Z 11849 listen of IPv6 address fails with EAFNOSUPPORT Reviewed by: Garrett D'Amore <gdamore@racktopsystems.com> Reviewed by: Dan McDonald <danmcd@joyent.com> Reviewed by: Toomas Soome <tsoome@me.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com>