Project

General

Profile

Bug #6588

SIOCLIFADDIFF takes no action if requested address can't be assigned

Added by Jorge Schrauwen over 4 years ago. Updated about 3 years ago.

Status:
New
Priority:
Normal
Assignee:
Dan McDonald
Category:
networking
Start date:
2016-01-30
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage
Gerrit CR:

Description

I was discussing some oddities in IPv6 under SmartOS today at FOSDEM with Dan.

I remember also having this on OmniOS in some cases when I had a ton of zones running on it.
We couldn't quickly replicate it but I managed to do so now, I rushed this a bit because Dan will have access to IPv6 connection tomorrow at his 2nd day at FOSDEM.

Below is the blurp form the e-mail I send him earler:

Hey Dan,

I can replicate it on OmniOS now. I am mailing this now because tomorrow at FOSDEM you will have access to working ipv6 internet.

It's because on SmartOS in.ntpd is not started via smf. (It may be some kind of race condition I guess)

Bellow are steps to replicate it, set IP to a IPv6 addr in the range.
It will create a vnic with allowed-ips set like on SmartOS (I was also setting this on OmniOS), it plumbs the interface, adds the static inet6 address, starts in.ntpd to grab the addrconf bits (else the static inet6 address won't work)

To simulate the late start of in.ntpd I kill it before creating the vnic. I tested this on a freshly instaleld OmniOS stable from the global zone.

I tested the small script below a few times (with a reboot, without a reboot, ...) as long as you cleanup the vnic you can keep running this to replicate it.

---
IP=2001:6f8:1480:10::123
NIC=vtest0
PARENT=e1000g0
pkill in.ndpd
dladm create-vnic -l ${PARENT} ${NIC}
dladm set-linkprop -p allowed-ips=${IP}/128 ${NIC}
ifconfig ${NIC} inet6 plumb up
ifconfig ${NIC} inet6 addif ${IP}/64 up
/usr/lib/inet/in.ndpd
---

Hopefully this is helpful, I will file a bug after I grab dinner and add you as watcher.

Regards

Jorge

History

#1

Updated by Jorge Schrauwen about 3 years ago

Easier way to reproduce this, it is probably lower down the stack ...

IP=2001:6f8:1480:10::123
IP_BAD=2001:6f8:1480:10::124
NIC=vtest0
PARENT=e1000g0
dladm create-vnic -l ${PARENT} ${NIC}
dladm set-linkprop -p allowed-ips=${IP}/128 ${NIC}
ifconfig ${NIC} inet6 plumb up
ifconfig ${NIC} inet6 addif ${IP}/64 up
ifconfig ${NIC} inet6 addif ${IP_BAD}/64 up

The 2nd ifconfig addif will fail:

Created new logical interface net0:3
ifconfig: could not create address:Permission denied

In the ifconfig example I believe the SIOCLIFADDR fails with eperm here https://github.com/illumos/illumos-gate/blob/f7877f5d39900cfd8b20dd673e5ccc1ef7cc7447/usr/src/cmd/cmd-inet/usr.sbin/ifconfig/ifconfig.c#L1963

Since we called SIOCLIFADDIF here https://github.com/illumos/illumos-gate/blob/f7877f5d39900cfd8b20dd673e5ccc1ef7cc7447/usr/src/cmd/cmd-inet/usr.sbin/ifconfig/ifconfig.c#L1888 we did not clean this up leaving the stray sub interface with the inet ::/0 address

Maybe something does not get cleaned up right for inet6, as doing the same with an inet address results in no such stray addresses.
This is where I hit a dead end because I couldn't figure out how to use dtrace to watch these iocls :(

I looked up SIOCLIFADDIF to see what it was doing but I didn't understand the ioctl code in ip.c, so not sure if it is responsible for the weird default address or not. But this is likely since it looks like both in.ndpd and ifconfig do both ioctl calls in the same order getting the same error.

Neither seem to be cleanup up the create if after the failure.

#2

Updated by Jorge Schrauwen about 3 years ago

Since I am not skilled enough to write a test program...

I had hoped to do this on both OmniOS and SmartOS, but ended up only doing SmartOS, I could not get illumos-gate compiled in my OmniOS KVM :(

base line

ifconfig output after boot:

[root@atom ~]# ifconfig
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
net0: flags=40201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS,L3PROTECT> mtu 1500 index 2
        inet 172.16.10.246 netmask ffffff00 broadcast 172.16.10.255
        ether 0:22:6:7:10:42
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128

Let's prepare our little experiment, we ifconfig net0 inet6 plumb up the interface

[root@atom ~]# ifconfig net0 inet6 plumb up
[root@atom ~]# ifconfig
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
net0: flags=40201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS,L3PROTECT> mtu 1500 index 2
        inet 172.16.10.246 netmask ffffff00 broadcast 172.16.10.255
        ether 0:22:6:7:10:42
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128
net0: flags=40202000841<UP,RUNNING,MULTICAST,IPv6,CoS,L3PROTECT> mtu 1500 index 2
        inet6 fe80::222:6ff:fe07:1042/10
        ether 0:22:6:7:10:42

Looking good!

the super ugly hack

I modified ifconfig's addif to stop after the SIOCLIFADDIF ioctl call...

--- smartos-live/projects/illumos/usr/src/cmd/cmd-inet/usr.sbin/ifconfig/ifconfig.c.orig        2017-03-27 20:53:39.393015245 +0000
+++ smartos-live/projects/illumos/usr/src/cmd/cmd-inet/usr.sbin/ifconfig/ifconfig.c     2017-03-27 20:56:43.899734568 +0000
@@ -1891,6 +1891,7 @@
        (void) printf("Created new logical interface %s\n",
            lifr.lifr_name);
        (void) strncpy(name, lifr.lifr_name, sizeof (name));
+        ipadmerr_exit(123, "HACK: error out after SIOCLIFADDIF");

        /*
         * Check and see if any "netmask" command is used and perform the

maybe SIOCLIFADDIF is the source of the problem?

Here is a truss from an unmodified ifconfig and the output after call addif:

[root@atom ~]# truss ifconfig net0 inet6 addif 2001:470:7ee7:10::198/64
execve("/sbin/ifconfig", 0x08047CF0, 0x08047D08)  argc = 5
sysinfo(SI_MACHINE, "i86pc", 257)               = 6
mmap(0x00000000, 32, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEFA0000
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEF90000
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEF80000
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEF70000
memcntl(0xFEFB5000, 49776, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
memcntl(0x08050000, 12524, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
resolvepath("/lib/ld.so.1", "/lib/ld.so.1", 1023) = 12
resolvepath("/sbin/ifconfig", "/sbin/ifconfig", 1023) = 14
sysconfig(_CONFIG_PAGESIZE)                     = 4096
stat64("/sbin/ifconfig", 0x08047970)            = 0
open("/var/ld/ld.config", O_RDONLY)             = 3
fstat64(3, 0x080474F0)                          = 0
mmap(0x00000000, 140, PROT_READ, MAP_SHARED, 3, 0) = 0xFEF60000
close(3)                                        = 0
stat64("/lib/libc.so.1", 0x08047190)            = 0
resolvepath("/lib/libc.so.1", "/lib/libc.so.1", 1023) = 14
open("/lib/libc.so.1", O_RDONLY)                = 3
mmapobj(3, MMOBJ_INTERPRET, 0xFEF70C38, 0x080471FC, 0x00000000) = 0
close(3)                                        = 0
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEE00000
memcntl(0xFEE10000, 261468, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
mmap(0x00010000, 24576, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEDF0000
getcontext(0x080477B0)
getrlimit(RLIMIT_STACK, 0x080477A8)             = 0
getpid()                                        = 34015 [34014]
lwp_private(0, 1, 0xFEDF2A40)                   = 0x000001C3
setustack(0xFEDF2AA0)
lwp_cond_broadcast(0xFEE000FC)                  = 0
lwp_cond_broadcast(0xFEF7062C)                  = 0
sysi86(SI86FPSTART, 0xFEF5BEAC, 0x0000133F, 0x00001F80) = 0x00000001
open("/etc/default/inet_type", O_RDONLY)        Err#2 ENOENT
sysconfig(_CONFIG_PAGESIZE)                     = 4096
stat64("/lib/libxnet.so.1", 0x08047228)         = 0
resolvepath("/lib/libxnet.so.1", "/lib/libxnet.so.1", 1023) = 17
open("/lib/libxnet.so.1", O_RDONLY)             = 3
mmapobj(3, MMOBJ_INTERPRET, 0xFEE00CE8, 0x08047294, 0x00000000) = 0
close(3)                                        = 0
stat64("/lib/libsocket.so.1", 0x08047068)       = 0
resolvepath("/lib/libsocket.so.1", "/lib/libsocket.so.1", 1023) = 19
open("/lib/libsocket.so.1", O_RDONLY)           = 3
mmapobj(3, MMOBJ_INTERPRET, 0xFEE00E98, 0x080470D4, 0x00000000) = 0
close(3)                                        = 0
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEDB0000
memcntl(0xFEDC0000, 17468, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
stat64("/lib/libnsl.so.1", 0x08046C38)          = 0
resolvepath("/lib/libnsl.so.1", "/lib/libnsl.so.1", 1023) = 16
open("/lib/libnsl.so.1", O_RDONLY)              = 3
mmapobj(3, MMOBJ_INTERPRET, 0xFEDB07C8, 0x08046CA4, 0x00000000) = 0
close(3)                                        = 0
memcntl(0xFEAE0000, 78532, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
sigfillset(0xFEF5B3E0)                          = 0
lwp_cond_broadcast(0xFEDB09AC)                  = 0
lwp_cond_broadcast(0xFEDB00FC)                  = 0
lwp_cond_broadcast(0xFEFFDD64)                  = 0
so_socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP, 0x00000000, SOV_XPG4_2) = 3
so_socket(PF_INET, SOCK_DGRAM, IPPROTO_IP, 0x00000000, SOV_XPG4_2) = 4
so_socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP, 0x00000000, SOV_XPG4_2) = 5
stat64("/lib/libipadm.so.1", 0x08047228)        = 0
resolvepath("/lib/libipadm.so.1", "/lib/libipadm.so.1", 1023) = 18
open("/lib/libipadm.so.1", O_RDONLY)            = 6
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEDA0000
mmapobj(6, MMOBJ_INTERPRET, 0xFEDA0018, 0x08047294, 0x00000000) = 0
close(6)                                        = 0
memcntl(0xFED70000, 20732, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
lwp_cond_broadcast(0xFEDA01FC)                  = 0
brk(0x00000000)                                 = 134669316
brk(0x0806E408)                                 = 0
brk(0x08070408)                                 = 0
so_socket(PF_INET, SOCK_DGRAM, IPPROTO_IP, 0x00000000, SOV_DEFAULT) = 6
so_socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP, 0x00000000, SOV_DEFAULT) = 7
zone_lookup(0x00000000)                         = 16
zone_getattr(16, ZONE_ATTR_FLAGS, 0x08047C2E, 2) = 2
stat64("/lib/libdladm.so.1", 0x080471D8)        = 0
resolvepath("/lib/libdladm.so.1", "/lib/libdladm.so.1", 1023) = 18
open("/lib/libdladm.so.1", O_RDONLY)            = 8
mmapobj(8, MMOBJ_INTERPRET, 0xFEDA07C8, 0x08047244, 0x00000000) = 0
close(8)                                        = 0
memcntl(0xFED30000, 50036, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
lwp_cond_broadcast(0xFEDA09AC)                  = 0
open("/dev/dld", O_RDWR)                        = 8
so_socket(PF_ROUTE, SOCK_RAW, 0, 0x00000000, SOV_DEFAULT) = 9
open("/etc/netconfig", O_RDONLY)                = 10
fstat64(10, 0x080465C0)                         = 0
fstat64(10, 0x080464C0)                         = 0
ioctl(10, TCGETA, 0x0804657E)                   Err#25 ENOTTY
read(10, " #   C D D L   H E A D E".., 2560)    = 2137
read(10, 0x0806E85C, 2560)                      = 0
llseek(10, 0, SEEK_CUR)                         = 2137
llseek(10, 0, SEEK_SET)                         = 0
read(10, " #   C D D L   H E A D E".., 2560)    = 2137
read(10, 0x0806E85C, 2560)                      = 0
llseek(10, 0, SEEK_CUR)                         = 2137
close(10)                                       = 0
open("/dev/udp", O_RDONLY)                      = 10
ioctl(10, SIOCGLIFNUM, 0x08046EA4)              = 0
close(10)                                       = 0
open("/dev/udp", O_RDONLY)                      = 10
ioctl(10, SIOCGLIFNUM, 0x08046EA4)              = 0
close(10)                                       = 0
brk(0x08070408)                                 = 0
brk(0x08072408)                                 = 0
ioctl(3, SIOCLIFADDIF, 0x0806DD00)              = 0
ioctl(1, TCGETA, 0x0804655E)                    = 0
fstat64(1, 0x080464A0)                          = 0
Created new logical interface net0:2
write(1, " C r e a t e d   n e w  ".., 37)      = 37
ioctl(3, SIOCGLIFGROUPNAME, 0x08047288)         = 0
stat64("/lib/libinetutil.so.1", 0x08046938)     = 0
resolvepath("/lib/libinetutil.so.1", "/lib/libinetutil.so.1", 1023) = 21
open("/lib/libinetutil.so.1", O_RDONLY)         = 10
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFED20000
mmapobj(10, MMOBJ_INTERPRET, 0xFED20018, 0x080469A4, 0x00000000) = 0
close(10)                                       = 0
memcntl(0xFED00000, 6460, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
lwp_cond_broadcast(0xFED201FC)                  = 0
getuid()                                        = 0 [0]
ioctl(6, SIOCGLIFFLAGS, 0x080471E8)             = 0
ioctl(6, SIOCGLIFGROUPNAME, 0x08047228)         = 0
ioctl(7, SIOCGLIFFLAGS, 0x08047228)             = 0
ioctl(6, SIOCGLIFFLAGS, 0x08047228)             = 0
open("/etc/svc/volatile/ipadm/ipmgmt_door", O_RDONLY) = 10
door_call(10, 0x080472C8)                       = 0
ioctl(7, SIOCGLIFFLAGS, 0x08047228)             = 0
door_call(10, 0x080471B8)                       = 0
door_call(10, 0x080471A8)                       = 0
open("/etc/svc/volatile/dladm/dlmgmt_door", O_RDONLY) = 11
door_call(11, 0x08046AC8)                       = 0
ioctl(7, SIOCGLIFFLAGS, 0x08047238)             = 0
ioctl(7, SIOCGLIFFLAGS, 0x08047288)             = 0
ioctl(7, SIOCSLIFNETMASK, 0x08047288)           = 0
ioctl(7, SIOCSLIFADDR, 0x08047288)              Err#1 EPERM
door_call(10, 0x08047358)                       = 0
fstat64(2, 0x08046B10)                          = 0
ifconfig: could not create address:write(2, " i f c o n f i g :   c o".., 35)   = 35
Permission deniedwrite(2, " P e r m i s s i o n   d".., 17)     = 17

write(2, "\n", 1)                               = 1
ioctl(3, SIOCLIFREMOVEIF, 0x08047AB8)           = 0
_exit(1)
[root@atom ~]# ifconfig
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
net0: flags=40201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS,L3PROTECT> mtu 1500 index 2
        inet 172.16.10.246 netmask ffffff00 broadcast 172.16.10.255
        ether 0:22:6:7:10:42
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128
net0: flags=40202000841<UP,RUNNING,MULTICAST,IPv6,CoS,L3PROTECT> mtu 1500 index 2
        inet6 fe80::222:6ff:fe07:1042/10
        ether 0:22:6:7:10:42
net0:1: flags=40202080840<RUNNING,MULTICAST,ADDRCONF,IPv6,CoS,L3PROTECT> mtu 1500 index 2
        inet6 ::/0
net0:2: flags=40202080840<RUNNING,MULTICAST,ADDRCONF,IPv6,CoS,L3PROTECT> mtu 1500 index 2
        inet6 ::/0

Hmm weird 2x inet6 ::/0, but OK we still see the problem.

Let's try out hacked ifconfig

[root@atom ~]# truss /root/smartos-live/proto/sbin/ifconfig net0 inet6 addif 2001:470:7ee7:10::198/64
execve("/root/smartos-live/proto/sbin/ifconfig", 0x08047CB8, 0x08047CD0)  argc = 5
sysinfo(SI_MACHINE, "i86pc", 257)               = 6
mmap(0x00000000, 32, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEFA0000
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEF90000
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEF80000
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEF70000
memcntl(0xFEFB5000, 49776, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
memcntl(0x08050000, 12524, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
resolvepath("/lib/ld.so.1", "/lib/ld.so.1", 1023) = 12
resolvepath("/root/smartos-live/proto/sbin/ifconfig", "/root/smartos-live/proto/sbin/ifconfig", 1023) = 38
sysconfig(_CONFIG_PAGESIZE)                     = 4096
stat64("/root/smartos-live/proto/sbin/ifconfig", 0x08047940) = 0
open("/var/ld/ld.config", O_RDONLY)             = 3
fstat64(3, 0x080474C0)                          = 0
mmap(0x00000000, 140, PROT_READ, MAP_SHARED, 3, 0) = 0xFEF60000
close(3)                                        = 0
stat64("/lib/libc.so.1", 0x08047160)            = 0
resolvepath("/lib/libc.so.1", "/lib/libc.so.1", 1023) = 14
open("/lib/libc.so.1", O_RDONLY)                = 3
mmapobj(3, MMOBJ_INTERPRET, 0xFEF70C38, 0x080471CC, 0x00000000) = 0
close(3)                                        = 0
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEE00000
memcntl(0xFEE10000, 261468, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
mmap(0x00010000, 24576, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEDF0000
getcontext(0x08047780)
getrlimit(RLIMIT_STACK, 0x08047778)             = 0
getpid()                                        = 34286 [34285]
lwp_private(0, 1, 0xFEDF2A40)                   = 0x000001C3
setustack(0xFEDF2AA0)
lwp_cond_broadcast(0xFEE000FC)                  = 0
lwp_cond_broadcast(0xFEF7062C)                  = 0
sysi86(SI86FPSTART, 0xFEF5BEAC, 0x0000133F, 0x00001F80) = 0x00000001
open("/etc/default/inet_type", O_RDONLY)        Err#2 ENOENT
sysconfig(_CONFIG_PAGESIZE)                     = 4096
stat64("/lib/libxnet.so.1", 0x080471E8)         = 0
resolvepath("/lib/libxnet.so.1", "/lib/libxnet.so.1", 1023) = 17
open("/lib/libxnet.so.1", O_RDONLY)             = 3
mmapobj(3, MMOBJ_INTERPRET, 0xFEE00CE8, 0x08047254, 0x00000000) = 0
close(3)                                        = 0
stat64("/lib/libsocket.so.1", 0x08047028)       = 0
resolvepath("/lib/libsocket.so.1", "/lib/libsocket.so.1", 1023) = 19
open("/lib/libsocket.so.1", O_RDONLY)           = 3
mmapobj(3, MMOBJ_INTERPRET, 0xFEE00E98, 0x08047094, 0x00000000) = 0
close(3)                                        = 0
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEDB0000
memcntl(0xFEDC0000, 17468, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
stat64("/lib/libnsl.so.1", 0x08046BF8)          = 0
resolvepath("/lib/libnsl.so.1", "/lib/libnsl.so.1", 1023) = 16
open("/lib/libnsl.so.1", O_RDONLY)              = 3
mmapobj(3, MMOBJ_INTERPRET, 0xFEDB07C8, 0x08046C64, 0x00000000) = 0
close(3)                                        = 0
memcntl(0xFEAE0000, 78532, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
sigfillset(0xFEF5B3E0)                          = 0
lwp_cond_broadcast(0xFEDB09AC)                  = 0
lwp_cond_broadcast(0xFEDB00FC)                  = 0
lwp_cond_broadcast(0xFEFFDD64)                  = 0
so_socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP, 0x00000000, SOV_XPG4_2) = 3
so_socket(PF_INET, SOCK_DGRAM, IPPROTO_IP, 0x00000000, SOV_XPG4_2) = 4
so_socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP, 0x00000000, SOV_XPG4_2) = 5
stat64("/lib/libipadm.so.1", 0x080471E8)        = 0
resolvepath("/lib/libipadm.so.1", "/lib/libipadm.so.1", 1023) = 18
open("/lib/libipadm.so.1", O_RDONLY)            = 6
mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEDA0000
mmapobj(6, MMOBJ_INTERPRET, 0xFEDA0018, 0x08047254, 0x00000000) = 0
close(6)                                        = 0
memcntl(0xFED70000, 20732, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
lwp_cond_broadcast(0xFEDA01FC)                  = 0
brk(0x00000000)                                 = 134669316
brk(0x0806E408)                                 = 0
brk(0x08070408)                                 = 0
so_socket(PF_INET, SOCK_DGRAM, IPPROTO_IP, 0x00000000, SOV_DEFAULT) = 6
so_socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP, 0x00000000, SOV_DEFAULT) = 7
zone_lookup(0x00000000)                         = 16
zone_getattr(16, ZONE_ATTR_FLAGS, 0x08047BEE, 2) = 2
stat64("/lib/libdladm.so.1", 0x08047198)        = 0
resolvepath("/lib/libdladm.so.1", "/lib/libdladm.so.1", 1023) = 18
open("/lib/libdladm.so.1", O_RDONLY)            = 8
mmapobj(8, MMOBJ_INTERPRET, 0xFEDA07C8, 0x08047204, 0x00000000) = 0
close(8)                                        = 0
memcntl(0xFED30000, 50036, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
lwp_cond_broadcast(0xFEDA09AC)                  = 0
open("/dev/dld", O_RDWR)                        = 8
so_socket(PF_ROUTE, SOCK_RAW, 0, 0x00000000, SOV_DEFAULT) = 9
open("/etc/netconfig", O_RDONLY)                = 10
fstat64(10, 0x08046580)                         = 0
fstat64(10, 0x08046480)                         = 0
ioctl(10, TCGETA, 0x0804653E)                   Err#25 ENOTTY
read(10, " #   C D D L   H E A D E".., 2560)    = 2137
read(10, 0x0806E85C, 2560)                      = 0
llseek(10, 0, SEEK_CUR)                         = 2137
llseek(10, 0, SEEK_SET)                         = 0
read(10, " #   C D D L   H E A D E".., 2560)    = 2137
read(10, 0x0806E85C, 2560)                      = 0
llseek(10, 0, SEEK_CUR)                         = 2137
close(10)                                       = 0
open("/dev/udp", O_RDONLY)                      = 10
ioctl(10, SIOCGLIFNUM, 0x08046E64)              = 0
close(10)                                       = 0
open("/dev/udp", O_RDONLY)                      = 10
ioctl(10, SIOCGLIFNUM, 0x08046E64)              = 0
close(10)                                       = 0
brk(0x08070408)                                 = 0
brk(0x08072408)                                 = 0
ioctl(3, SIOCLIFADDIF, 0x0806DD00)              = 0
ioctl(1, TCGETA, 0x0804651E)                    = 0
fstat64(1, 0x08046460)                          = 0
Created new logical interface net0:3
write(1, " C r e a t e d   n e w  ".., 37)      = 37
fstat64(2, 0x08046430)                          = 0
ifconfig: write(2, " i f c o n f i g :  ", 10)          = 10
HACK: error out after SIOCLIFADDIFwrite(2, " H A C K :   e r r o r  ".., 34)    = 34
: write(2, " :  ", 2)                           = 2
<unknown error>write(2, " < u n k n o w n   e r r".., 15)       = 15

write(2, "\n", 1)                               = 1
_exit(1)
[root@atom ~]# ifconfig
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
net0: flags=40201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS,L3PROTECT> mtu 1500 index 2
        inet 172.16.10.246 netmask ffffff00 broadcast 172.16.10.255
        ether 0:22:6:7:10:42
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128
net0: flags=40202000841<UP,RUNNING,MULTICAST,IPv6,CoS,L3PROTECT> mtu 1500 index 2
        inet6 fe80::222:6ff:fe07:1042/10
        ether 0:22:6:7:10:42
net0:1: flags=40202080840<RUNNING,MULTICAST,ADDRCONF,IPv6,CoS,L3PROTECT> mtu 1500 index 2
        inet6 ::/0
net0:2: flags=40202080840<RUNNING,MULTICAST,ADDRCONF,IPv6,CoS,L3PROTECT> mtu 1500 index 2
        inet6 ::/0
net0:3: flags=40202000840<RUNNING,MULTICAST,IPv6,CoS,L3PROTECT> mtu 1500 index 2
        inet6 ::/0

net0:3: flags=40202000840<RUNNING,MULTICAST,IPv6,CoS,L3PROTECT> mtu 1500 index 2
        inet6 ::/0

Looks like we have the same problem, as you can see from the truss we only called SIOCLIFADDIF.
This seems to point that the bug is probably somewhere in the handling of the SIOCLIFADDIF ioctl... but I don't understand ip.c/ip_if.c at all, I had a look but I can't even figure out where the ioctl is implemented there :(

This also explains both ifconfig and in.ndpd have the issues, both of these use the same ioctl calls.

Ideally I wanted to write a small self contain program but E_SKILL_MISSING was encountered ;) Neither could I get this data out of dtrace/mdb... although I am sure one of those would have been able to help without doing this ugly hack.

#3

Updated by Yuri Pankov about 3 years ago

We start here in ip_process_ioctl() with SIOCIFADDIF being special case:

http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/inet/ip/ip.c#12573

So we are now in ip_sioctl_addif(), interface is successfully created, now to set address:

http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/inet/ip/ip_if.c#9419

And we are in ip_sioctl_addr() checking allowed IPs and returning with EPERM in your case:

http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/inet/ip/ip_if.c#9754

The error gets propagated back to ip_process_ioctl(), which calls ip_ioctl_finish(), which too doesn't take any specific action on that error, so we happily mi_copy_done(), setting ioc_error to the value of err.

Kernel part is done, we are back with created interface and non-created address, so ::/0.

I don't think it's a bug per se, whether special handling of EPERM is needed by userland consumers is another question.

#4

Updated by Yuri Pankov about 3 years ago

  • Subject changed from in.ndpd can add 'inet6 ::/0' in some cases to SIOCLIFADDIFF takes no action if requested address can't be assigned

Also available in: Atom PDF