Project

General

Profile

Actions

Feature #13500

closed

Want support for overlay networks

Added by Andy Fiddaman over 1 year ago. Updated 3 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
networking
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Hard
Tags:
Gerrit CR:

Description

This is the master ticket for tracking the upstreaming of the network overlay feature from SmartOS:

See https://github.com/illumos/ipd/blob/master/ipd/0018/README.md

The commits for this work can be found in https://github.com/citrus-it/illumos-omnios/tree/ig_overlay


Related issues

Related to illumos gate - Bug #14436: "dladm show-overlay <overlay>" exits zero when varpd doesn't know about the overlayClosedAndy Fiddaman

Actions
Related to illumos gate - Feature #14434: overlays should persist across rebootsClosedAndy Fiddaman

Actions
Related to illumos gate - Bug #14435: overlay encapsulation bind failure error message is unhelpfulClosedAndy Fiddaman

Actions
Related to illumos gate - Bug #14437: cyclics corked on overlay socket with full queueNewAndy Fiddaman

Actions
Related to illumos gate - Feature #14444: Convert dladm(1m) to mdocIn ProgressAndy Fiddaman

Actions
Related to illumos gate - Bug #14538: varpd is noisy on consoleClosedAndy Fiddaman

Actions
Related to illumos gate - Feature #4186: Add VXLAN support to illumosResolvedRobert Mustacchi2013-10-07

Actions
Blocked by illumos gate - Feature #14196: Want librenameClosedAndy Fiddaman

Actions
Blocked by illumos gate - Feature #14197: Implement id_space as a libraryClosedAndy Fiddaman

Actions
Blocked by illumos gate - Bug #14198: stack overflow from promisc callbacksClosedAndy Fiddaman

Actions
Blocked by illumos gate - Bug #14199: sendfile compat checks shouldn't be done in so_sendmblkClosedAndy Fiddaman

Actions
Blocked by illumos gate - Feature #14200: refhash could be used outside of mpt_sasClosedAndy Fiddaman

Actions
Blocked by illumos gate - Feature #14201: want string property ranges for macClosedAndy Fiddaman

Actions
Blocked by illumos gate - Feature #14202: Need direct callbacks from socket upcalls via ksocketClosedAndy Fiddaman

Actions
Blocked by illumos gate - Feature #14203: snoop should support vxlanClosedAndy Fiddaman

Actions
Blocked by illumos gate - Bug #14204: mac_rx_srs_process stack depth needs to account for harder usageClosedAndy Fiddaman

Actions
Blocked by illumos gate - Feature #14205: Want UDP src port hashing for VXLANClosedAndy Fiddaman

Actions
Actions #1

Updated by Andy Fiddaman over 1 year ago

  • Description updated (diff)
Actions #2

Updated by Andy Fiddaman over 1 year ago

  • Subject changed from Upstream network overlay feature to gate to I for one, welcome my overlay network overlords
Actions #3

Updated by Andy Fiddaman over 1 year ago

  • Description updated (diff)
Actions #4

Updated by Andy Fiddaman 10 months ago

  • Subject changed from I for one, welcome my overlay network overlords to Want support for "overlay" networks
Actions #5

Updated by Andy Fiddaman 7 months ago

Actions #6

Updated by Andy Fiddaman 7 months ago

Actions #7

Updated by Andy Fiddaman 7 months ago

  • Blocked by Bug #14198: stack overflow from promisc callbacks added
Actions #8

Updated by Andy Fiddaman 7 months ago

  • Blocked by Bug #14199: sendfile compat checks shouldn't be done in so_sendmblk added
Actions #9

Updated by Andy Fiddaman 7 months ago

  • Blocked by Feature #14200: refhash could be used outside of mpt_sas added
Actions #10

Updated by Andy Fiddaman 7 months ago

  • Blocked by Feature #14201: want string property ranges for mac added
Actions #11

Updated by Andy Fiddaman 7 months ago

  • Blocked by Feature #14202: Need direct callbacks from socket upcalls via ksocket added
Actions #12

Updated by Andy Fiddaman 7 months ago

Actions #13

Updated by Andy Fiddaman 7 months ago

  • Blocked by Bug #14204: mac_rx_srs_process stack depth needs to account for harder usage added
Actions #14

Updated by Andy Fiddaman 7 months ago

  • Blocked by Feature #14205: Want UDP src port hashing for VXLAN added
Actions #15

Updated by Andy Fiddaman 7 months ago

  • Description updated (diff)
Actions #16

Updated by Electric Monk 4 months ago

  • Gerrit CR set to 1972
Actions #17

Updated by Andy Fiddaman 4 months ago

  • Subject changed from Want support for "overlay" networks to Want support for overlay networks
Actions #18

Updated by Andy Fiddaman 4 months ago

Actions #19

Updated by Andy Fiddaman 4 months ago

  • Blocks Bug #14435: overlay encapsulation bind failure error message is unhelpful added
Actions #20

Updated by Andy Fiddaman 4 months ago

  • Related to Bug #14436: "dladm show-overlay <overlay>" exits zero when varpd doesn't know about the overlay added
Actions #21

Updated by Andy Fiddaman 4 months ago

  • Blocks deleted (Feature #14434: overlays should persist across reboots)
Actions #22

Updated by Andy Fiddaman 4 months ago

  • Blocks deleted (Bug #14435: overlay encapsulation bind failure error message is unhelpful)
Actions #23

Updated by Andy Fiddaman 4 months ago

  • Related to Feature #14434: overlays should persist across reboots added
Actions #24

Updated by Andy Fiddaman 4 months ago

  • Related to Bug #14435: overlay encapsulation bind failure error message is unhelpful added
Actions #25

Updated by Andy Fiddaman 4 months ago

  • Related to Bug #14437: cyclics corked on overlay socket with full queue added
Actions #26

Updated by Andy Fiddaman 3 months ago

Actions #27

Updated by Andy Fiddaman 3 months ago

I have tested this change by creating temporary overlays where the remote end point is one or more OmniOS or SmartOS instances, testing with both the 'direct' and 'files' search plugins. I've also tested between two boxes ONUd to these same bits.

For example, a simple point-to-point overlay can be created as follows:

dladm create-overlay -t \
        -v 142 \
        -e vxlan \
        -s direct \
        -p vxlan/listen_ip=172.27.10.9 \
        -p vxlan/listen_port=4789 \
        -p direct/dest_ip=10.1.0.1 \
        -p direct/dest_port=4789 \
        ov0

dladm create-vnic -t -l ov0 testv0
ipadm create-if -t testv0
ipadm create-addr -t -T static -a local=10.20.30.1/24 testv0/v4

% ssh af@10.20.30.2
(af@10.20.30.2) Password:
OmniOS r151040  omnios-r151040-3239bd100b       January 2022
theeo%

I have tested the files search plugin by linking it into my own overlay network which spans several hosts, but for specific further tests I used this small configuration file:

{
        "2:8:20:f2:b2:70": {
                "arp": "10.20.30.2",
                "ip": "192.168.0.2",
                "port": 4789
        },
        "de:ad:be:ef:00:01": {
                "arp": "10.20.30.4",
                "ip": "192.168.0.1",
                "port": 4789
        }
}

and an overlay created with:

dladm create-overlay -t \
        -v 142 \
        -e vxlan \
        -s files \
        -p vxlan/listen_ip=172.27.10.9 \
        -p vxlan/listen_port=4789 \
        -p files/config=/data/omnios-build/ovtest/config.json \
        ov0
Actions #28

Updated by Electric Monk 3 months ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 36589d6bb0cdae89e166b57b0d64ae56d53247d9

commit  36589d6bb0cdae89e166b57b0d64ae56d53247d9
Author: Robert Mustacchi <rm@joyent.com>
Date:   2022-02-22T10:37:31.000Z

    13500 Want support for overlay networks
    Portions contributed by: Dan McDonald <danmcd@joyent.com>
    Portions contributed by: Jason King <jason.king@joyent.com>
    Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
    Portions contributed by: Mike Zeller <mike@mikezeller.net>
    Portions contributed by: Andy Fiddaman <omnios@citrus-it.co.uk>
    Reviewed by: Andy Fiddaman <andy@omnios.org>
    Reviewed by: Dan McDonald <danmcd@joyent.com>
    Reviewed by: Jason King <jbk@joyent.com>
    Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
    Reviewed by: Joshua M. Clulow <jmc@joyent.com>
    Reviewed by: Mike Gerdts <mike.gerdts@joyent.com>
    Reviewed by: Mike Zeller <mike.zeller@joyent.com>
    Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
    Reviewed by: Rob Gulewich <robert.gulewich@joyent.com>
    Reviewed by: Robert Mustacchi <rm@joyent.com>
    Reviewed by: Ryan Zezeski <rpz@joyent.com>
    Approved by: Joshua M. Clulow <josh@sysmgr.org>

Actions #29

Updated by Andy Fiddaman 3 months ago

  • Related to Bug #14538: varpd is noisy on console added
Actions #30

Updated by Robert Mustacchi 21 days ago

Actions

Also available in: Atom PDF