Q-in-Q: IEEE 802.1ad Shared Core Network

Goals

IEEE 802.1ad, also known as Q-in-Q or provider bridging, lets a network carry the VLANs of several customers without mixing them, even if the customers use the same VLAN ids. The provider’s bridges put an extra VLAN tag, the S-tag, in front of the customer’s own tag, and forward the frames by the S-tag only.

This showcase presents the wholesale case: a core network shared by two service providers. Each service provider runs C-VLANs for its own customers, and both of them use the same C-VLAN ids. To make the point even clearer, the hosts of the second service provider use the same MAC addresses as the hosts of the first one. The showcase demonstrates that the core still keeps the two service providers apart, because it relays the frames by the S-VLAN of the service provider and never looks at the C-tags.

Verified with INET version: 4.8
Source files location: inet/showcases/ethernet/sharedcore

About Q-in-Q

IEEE 802.1ad, now part of IEEE 802.1Q, defines two kinds of VLAN tags. Both have the familiar four-byte layout: a Tag Protocol Identifier (TPID) in the place of the EtherType, followed by the Priority Code Point (PCP), the Drop Eligible Indicator (DEI) and the VLAN id. Only the TPID tells them apart:

  • The C-tag (customer tag, TPID 0x8100) is the ordinary VLAN tag. A network that uses C-tags is built from C-VLAN Bridges.

  • The S-tag (service tag, TPID 0x88a8) is the tag of the provider. A network that uses S-tags is built from S-VLAN Bridges.

When a frame enters the provider’s network, a bridge of the provider pushes an S-tag in front of the C-tag, if the frame has one. Inside the provider’s network, the frame looks like this on the wire:

| destination MAC | source MAC | S-tag 0x88a8 | C-tag 0x8100 | EtherType | payload | FCS |

An S-VLAN Bridge reads and writes only the S-tag. The C-tag is part of the payload for it, so the C-VLAN ids of the customers are completely invisible to the provider’s network; they can overlap between customers.

The interface where the provider takes over a customer’s frames is called a service interface. In the port-based service interface, the bridge of the provider puts all frames that arrive on the port of a customer into the S-VLAN of that customer, whatever C-tag they have. The port is then a Customer Network Port. (IEEE 802.1Q also defines a Provider Edge Bridge, which has C-VLAN components besides the S-VLAN component and can select the S-VLAN by the C-VLAN of the frame; this showcase does not need one.)

Inside the provider’s network, the ports that connect its bridges to each other carry the frames of several S-VLANs, S-tagged; they are Provider Network Ports.

VLAN-aware bridges learn MAC addresses per VLAN: a MAC forwarding table entry is a (VLAN id, MAC address) pair. The same MAC address can thus appear in several VLANs, on different ports, without the entries interfering. This is why the reused MAC addresses in this showcase cause no harm in the core: the two copies of each address are learned in different S-VLANs.

The Model

The Network

The network contains two service providers, each with a network in the west and one in the east, and the shared core between them:

../../src/media/showcases/ethernet/sharedcore/doc/network.png

On the canvas, the networks of service provider 1 are blue, those of service provider 2 orange, and the core network green; each customer site has a color of its own. The labels on the links between bridges name the VLANs that the link carries tagged.

The nodes play the following roles:

  • Hosts (host*) are StandardHosts without an 802.1Q layer. They send and receive untagged frames.

  • Customer switches (switchB1, switchC1, switchA2, switchD2) are VLAN-unaware EthernetSwitches.

  • Service provider bridges (sp1w, sp1e, sp2w, sp2e) are C-VLAN Bridges. They put each customer into a C-VLAN, 10 for customers A and C and 20 for customers B and D, and send the frames C-tagged to the core.

  • Edge bridges of the core (pebW, pebE) are S-VLAN Bridges whose ports toward the service providers are Customer Network Ports: they put all frames of service provider 1 into S-VLAN 100, and all frames of service provider 2 into S-VLAN 200. Their port toward the core bridge is a Provider Network Port. Despite their names, they are not Provider Edge Bridges in the IEEE 802.1Q sense, as they have no C-VLAN components.

  • The core bridge (pb) is an S-VLAN Bridge that relays the frames by S-VLAN. Both of its ports are Provider Network Ports.

Note the point of view: for the core network, the customers are the two service providers. Service provider 1 has customers A and B, service provider 2 has customers C and D. Each customer has a site in the west and one in the east. A customer site is either two hosts on access ports of the service provider’s bridge, or a customer switch with two hosts, connected to one access port; every customer has one site of each kind.

Ports of the Bridges

IEEE 802.1Q describes the VLAN behavior of a bridge by a few settings of each of its ports:

  • The frames that the port admits: untagged and priority-tagged frames, VLAN-tagged frames, or both. A priority-tagged frame has a tag with VLAN id 0, which carries a priority but no VLAN.

  • The VLAN of the port (PVID): the port classifies the untagged and priority-tagged frames that it admits into this VLAN.

  • The VLANs that the port is a member of: the bridge accepts the frames of these VLANs on the port, and sends the frames of a VLAN only to the ports that are members of it.

  • The VLANs that the port sends untagged; it sends the frames of its other VLANs tagged.

The tags in these settings are the tags of the bridge: C-tags in a C-VLAN Bridge, S-tags in an S-VLAN Bridge. For an S-VLAN Bridge, a frame with a C-tag but no S-tag is untagged.

The VLAN-aware bridges of this showcase use four combinations of these settings, the port types:

  • An access port is a member of one VLAN, the VLAN of the port. It admits untagged and priority-tagged frames only, and sends the frames of its VLAN untagged.

  • A trunk port is a member of several VLANs. It admits VLAN-tagged frames only, and sends all frames tagged.

  • A Customer Network Port is an access port of an S-VLAN Bridge: it puts the frames that arrive without an S-tag into the S-VLAN of the customer, whatever C-tag they have, and sends the frames of that S-VLAN without an S-tag.

  • A Provider Network Port is a trunk port of an S-VLAN Bridge.

The configuration refers to the ports by their interface names. The ports are numbered in the order of the connections in the NED file:

  • A customer switch has its hosts on eth0 and eth1, and the uplink to the service provider on eth2. It is VLAN-unaware, so its ports have no type.

  • A service provider bridge has its customer sites on access ports, eth0 to eth2, and its uplink to the core on a trunk port, eth3, which carries both C-VLANs.

  • pebW and pebE have service provider 1 on eth0 and service provider 2 on eth1, both Customer Network Ports, and pb on eth2, a Provider Network Port that carries both S-VLANs.

  • pb has pebW on eth0 and pebE on eth1, both Provider Network Ports that carry both S-VLANs.

Inside the Bridges

All VLAN-aware bridges of the network, the service provider bridges and the bridges of the core, have the same internal structure. The following image shows the inside of pebW:

../../src/media/showcases/ethernet/sharedcore/doc/pebW.png

The network interfaces (eth[*]) are at the bottom, and the protocol layers are connected to each other through the li and bl message dispatchers. A frame received by an interface goes up to the ethernet layer, which checks and removes the Ethernet MAC header. The dispatchers then send the frame on by the protocol of its payload. If the payload starts with a tag that the ieee8021q layer handles, the frame goes there; the layer removes the tag, and passes its contents (VLAN id, PCP and DEI) up as packet tags. Every other frame goes directly to the bridging layer. On the way down, the ieee8021q layer inserts a tag if the bridging layer requested one, and the ethernet layer adds the MAC header. The cb and sc dispatchers above the bridging layer connect it to the protocols of the bridge itself, such as the Spanning Tree Protocol; none of them runs in this showcase.

The 802.1Q layer handles one kind of tag only, the C-tag or the S-tag; this is selected by its vlanTagType parameter. In pebW, it handles the S-tag. A frame that arrives from a service provider has a C-tag but no S-tag, so it bypasses the 802.1Q layer, and the C-tag stays in the payload of the frame.

The bridging layer is a BridgingLayer:

../../src/media/showcases/ethernet/sharedcore/doc/pebW_bridging.png

From the top down, the submodules that this showcase uses are the following:

  • directionReverser (DirectionReverserLayer) decides whether a frame is delivered locally or forwarded. For a forwarded frame, it turns the indication tags of the received frame into request tags for the outgoing frame, for example the received VLAN id into the requested one.

  • interfaceRelay (InterfaceRelayLayer) learns the source MAC address of each received frame in the MAC forwarding table, per VLAN, and selects the outgoing port of each frame by its destination MAC address and VLAN. If the destination is unknown, it sends the frame to all other ports.

  • vlanPolicy (Ieee8021adPolicy or VlanPolicyLayer) applies the port settings described in Ports of the Bridges: it drops the frames that a port does not admit, classifies the untagged frames into the VLAN of the port, keeps the frames of a VLAN away from the ports that are not its members, and decides whether a frame leaves a port tagged or untagged.

BridgingLayer also has slots for stream identification, stream relay, stream filtering, stream coding and processing delay, which the Time-Sensitive Networking features of INET use. They are empty here, so they do not appear in the image.

The showcase has two configurations, which differ only in the VLAN policy of the bridges. In the PortTypes configuration, which the rest of this page follows, vlanPolicy is an Ieee8021adPolicy, a single module that is configured by the type of each port. In the FiltersAndMappers configuration, it is a VlanPolicyLayer, which consists of VLAN filters and mappers; see The Same Bridges with Filters and Mappers.

The same structure serves C-VLAN Bridges and S-VLAN Bridges: the kind of the bridge depends only on the vlanTagType parameter of its modules. The customer switches, in contrast, have no 802.1Q layer, and their bridging layer is the default Ieee8021dRelay.

Addresses and Traffic

Each customer is an IP subnet of its own, configured by the Ipv4NetworkConfigurator:

<config>
  <interface hosts="hostA*" address="10.1.10.x" netmask="255.255.255.0"/>
  <interface hosts="hostB*" address="10.1.20.x" netmask="255.255.255.0"/>
  <interface hosts="hostC*" address="10.2.10.x" netmask="255.255.255.0"/>
  <interface hosts="hostD*" address="10.2.20.x" netmask="255.255.255.0"/>
</config>

Every host pings its counterpart at the other site of the same customer five times, 10 ms apart. hostA1a starts at 1 ms, so that its first ping crosses the network alone (see The First Ping), and the other hosts start at 2 ms. This is 80 ping requests altogether within the 100 ms of simulated time.

Configuration

The hosts have no 802.1Q layer, so they send and receive untagged frames. Their MAC addresses are set explicitly: customer C uses the MAC addresses of customer A, and customer D those of customer B:

# The hosts send and receive untagged frames; they have no 802.1Q layer. The
# hosts of service provider 2 use the MAC addresses of service provider 1's
# hosts, customer C those of customer A and customer D those of customer B.
*.hostA1a.eth[0].address = "02-00-00-00-0a-1a"
*.hostA1b.eth[0].address = "02-00-00-00-0a-1b"
*.hostA2a.eth[0].address = "02-00-00-00-0a-2a"
*.hostA2b.eth[0].address = "02-00-00-00-0a-2b"
*.hostB1a.eth[0].address = "02-00-00-00-0b-1a"
*.hostB1b.eth[0].address = "02-00-00-00-0b-1b"
*.hostB2a.eth[0].address = "02-00-00-00-0b-2a"
*.hostB2b.eth[0].address = "02-00-00-00-0b-2b"
*.hostC1a.eth[0].address = "02-00-00-00-0a-1a"
*.hostC1b.eth[0].address = "02-00-00-00-0a-1b"
*.hostC2a.eth[0].address = "02-00-00-00-0a-2a"
*.hostC2b.eth[0].address = "02-00-00-00-0a-2b"
*.hostD1a.eth[0].address = "02-00-00-00-0b-1a"
*.hostD1b.eth[0].address = "02-00-00-00-0b-1b"
*.hostD2a.eth[0].address = "02-00-00-00-0b-2a"
*.hostD2b.eth[0].address = "02-00-00-00-0b-2b"

The service provider bridges and the bridges of the core are VLAN-aware. Each of them gets an 802.1Q layer and a BridgingLayer. The customer switches keep their defaults, so they remain VLAN-unaware:

# The customer switches are VLAN-unaware. All other bridges are VLAN-aware:
# they learn and forward per VLAN, in the VLAN space of their 802.1Q layer.
*.sp*.ieee8021q.typename = "Ieee8021qProtocol"
*.peb*.ieee8021q.typename = "Ieee8021qProtocol"
*.pb.ieee8021q.typename = "Ieee8021qProtocol"
*.sp*.bridging.typename = "BridgingLayer"
*.peb*.bridging.typename = "BridgingLayer"
*.pb.bridging.typename = "BridgingLayer"

The service provider bridges are C-VLAN Bridges, as the vlanTagType parameter is "c" by default. A single line per node makes the bridges of the core S-VLAN Bridges: setting vlanTagType to "s" on all modules of the node makes the 802.1Q layer insert and remove S-tags, and makes the VLAN policy and the relay work with the S-VLAN of the frames:

# The bridges of the core are S-VLAN bridges: their 802.1Q layer inserts and
# removes S-tags, and leaves the C-tags of the service providers in the
# payload, and their VLAN policy and relay work with the S-VLAN of the frames.
*.peb*.**.vlanTagType = "s"
*.pb.**.vlanTagType = "s"

pebW records the traffic of its port toward service provider 1 and of its port toward the core bridge into PCAP files:

# The port of pebW toward service provider 1 (eth0) carries C-tagged frames,
# its port toward the core bridge (eth2) S-tagged frames over C-tagged ones.
*.pebW.numPcapRecorders = 2
*.pebW.pcapRecorder[0].moduleNamePatterns = "eth[0]"
*.pebW.pcapRecorder[0].pcapFile = "${resultdir}/${configname}-#${runnumber}-pebW_eth0.pcap"
*.pebW.pcapRecorder[1].moduleNamePatterns = "eth[2]"
*.pebW.pcapRecorder[1].pcapFile = "${resultdir}/${configname}-#${runnumber}-pebW_eth2.pcap"

These settings are in the abstract [General] section, which both configurations extend. The PortTypes configuration makes the vlanPolicy of every VLAN-aware bridge an Ieee8021adPolicy:

[Config PortTypes]
description = "two service providers, each with C-VLANs for its customers, share a core that separates them by S-VLAN; the bridge ports are configured by port type with Ieee8021adPolicy"

# The VLAN policy of each bridge gives each of its ports an IEEE 802.1Q port
# type. The direction reverser forwards the PCP (the default): a port that sends
# a VLAN untagged sends it without any tag.
*.sp*.bridging.vlanPolicy.typename = "Ieee8021adPolicy"
*.peb*.bridging.vlanPolicy.typename = "Ieee8021adPolicy"
*.pb.bridging.vlanPolicy.typename = "Ieee8021adPolicy"

The ports parameter of Ieee8021adPolicy maps the name of each interface to its port: its type, and the VLAN of an access port or Customer Network Port (vid), or the VLANs of a trunk port or Provider Network Port (vids). The service provider bridges have an access port in the C-VLAN of the customer for each customer site, and a trunk port with both C-VLANs toward the core. The sites are on different ports in each bridge, so each bridge has a line of its own:

# The bridges of the service providers are C-VLAN bridges. The ports of the
# customers are access ports: they accept untagged and priority-tagged frames
# and classify them into the C-VLAN of the customer. The uplink to the core is
# a trunk that carries both C-VLANs C-tagged. A customer switch takes one access
# port, two hosts take two.
*.sp1w.bridging.vlanPolicy.ports = {"eth0" : {"type" : "access", "vid" : 10}, "eth1" : {"type" : "access", "vid" : 10}, "eth2" : {"type" : "access", "vid" : 20}, "eth3" : {"type" : "trunk", "vids" : [10, 20]}}
*.sp1e.bridging.vlanPolicy.ports = {"eth0" : {"type" : "access", "vid" : 10}, "eth1" : {"type" : "access", "vid" : 20}, "eth2" : {"type" : "access", "vid" : 20}, "eth3" : {"type" : "trunk", "vids" : [10, 20]}}
*.sp2w.bridging.vlanPolicy.ports = {"eth0" : {"type" : "access", "vid" : 10}, "eth1" : {"type" : "access", "vid" : 20}, "eth2" : {"type" : "access", "vid" : 20}, "eth3" : {"type" : "trunk", "vids" : [10, 20]}}
*.sp2e.bridging.vlanPolicy.ports = {"eth0" : {"type" : "access", "vid" : 10}, "eth1" : {"type" : "access", "vid" : 10}, "eth2" : {"type" : "access", "vid" : 20}, "eth3" : {"type" : "trunk", "vids" : [10, 20]}}

The frames that a service provider bridge forwards from an access port to another access port in the same C-VLAN stay untagged, and no frame leaves on an access port of another C-VLAN.

The edge bridges of the core implement the port-based service interface: a Customer Network Port toward each service provider, in the S-VLAN of that service provider, and a Provider Network Port with both S-VLANs toward the core bridge. The two edge bridges have the same ports, so one line configures both:

# An edge bridge of the core has a customer network port (cnp) toward each
# service provider: it accepts frames without an S-tag, puts all of them into
# the S-VLAN of that service provider, and removes the S-tag when it sends to
# it. Its provider network port (pnp) toward the core bridge carries both
# S-VLANs S-tagged.
*.peb*.bridging.vlanPolicy.ports = {"eth0" : {"type" : "cnp", "vid" : 100}, "eth1" : {"type" : "cnp", "vid" : 200}, "eth2" : {"type" : "pnp", "vids" : [100, 200]}}

The core bridge has two Provider Network Ports with both S-VLANs:

# The core bridge relays by S-VLAN only.
*.pb.bridging.vlanPolicy.ports = {"eth0" : {"type" : "pnp", "vids" : [100, 200]}, "eth1" : {"type" : "pnp", "vids" : [100, 200]}}

To sum up, a Q-in-Q network with a port-based service interface takes the following settings in INET:

  • Every VLAN-aware bridge gets an Ieee8021qProtocol as its ieee8021q layer, a BridgingLayer as its bridging layer, and an Ieee8021adPolicy as the vlanPolicy of the latter.

  • The bridges of the provider are S-VLAN Bridges: their vlanTagType is "s". The C-VLAN Bridges in front of them, here the bridges of the service providers, keep the default, "c".

  • The ports of the bridges of the provider are Customer Network Ports (cnp) toward the customers, and Provider Network Ports (pnp) between each other.

A Frame’s Path through an Edge Bridge

This section follows a ping request of customer A through pebW, from service provider 1 toward the core bridge, using the module structure shown in Inside the Bridges.

  1. The frame arrives on eth0 from sp1w, C-tagged with C-VLAN 10. The ethernet layer removes the MAC header, and the dispatchers send the frame directly to the bridging layer, because it has no S-tag.

  2. The frame has no S-tag, so it is untagged for the vlanPolicy. eth0 is a Customer Network Port, which admits untagged frames and classifies them into its VLAN, S-VLAN 100.

  3. The learner of the interfaceRelay learns the source MAC address of the frame on eth0 in S-VLAN 100.

  4. The directionReverser forwards the frame, and turns its S-VLAN indication into an S-VLAN request.

  5. The selector of the interfaceRelay looks up the destination MAC address in S-VLAN 100, and selects eth2, the port toward pb.

  6. eth2 is a member of S-VLAN 100, and sends it tagged, so the vlanPolicy lets the frame leave there with its S-VLAN request.

  7. Because the frame has an S-VLAN request, the dispatchers send it to the ieee8021q layer, which inserts an S-tag with VLAN id 100. The C-tag is now behind the S-tag. The ethernet layer adds the MAC header with the S-tag’s TPID, 0x88a8, and the frame leaves on eth2.

The ping reply takes the opposite path. It arrives on eth2 with an S-tag, so the dispatchers send it to the ieee8021q layer, which removes the S-tag and indicates S-VLAN 100. eth0 sends S-VLAN 100 untagged, so the vlanPolicy removes the S-VLAN request, the frame bypasses the 802.1Q layer on the way down, and it leaves eth0 with its C-tag only.

Note that the C-tag is never touched in pebW: neither the VLAN policy nor the relay look at it.

The Same Bridges with Filters and Mappers

The port types of Ieee8021adPolicy cover the ports of this showcase, but not every possible port configuration: they have no VLAN id translation, and no hybrid ports, which send some of their VLANs tagged and others untagged. VlanPolicyLayer can express these too. It consists of VLAN filters and mappers that are configured per port and per VLAN id, and INET also uses it in the 802.1Q layer of hosts.

The FiltersAndMappers configuration builds the ports of this showcase from the filters and mappers of VlanPolicyLayer. In this network, it produces the same frames as PortTypes: the PCAP files of pebW are byte-identical in the two configurations, and so are the ping results. The vlanPolicy module of pebW looks like this in the FiltersAndMappers configuration:

../../src/media/showcases/ethernet/sharedcore/doc/pebW_vlanpolicy.png

On the way in, the inboundFilter drops the frames whose VLAN id the incoming port does not accept, and then the inboundMapper can replace the VLAN id, for example to put the untagged frames of a port into a VLAN. On the way out, the outboundMapper can replace or remove the VLAN id per outgoing port, and then the outboundFilter drops the frames whose VLAN id the outgoing port does not accept. Both filters check the VLAN id as it is on the wire: -1 stands for an untagged frame, and 0 for a priority-tagged one.

The filters and the mappers treat a port without an entry differently. A filter drops all frames on a port that it has no entry for, so its list must name every port of the bridge. A mapper passes the frames of such a port unchanged, and also the frames whose VLAN id has no entry in the map of the port, so it needs entries only where it changes the VLAN id.

A port type thus becomes entries in the filters and mappers:

  • An access port or a Customer Network Port accepts untagged frames (-1) in both filters, has an inbound mapper entry from -1 to the VLAN of the port, and an outbound mapper entry from that VLAN to -1. The inbound mapper is a VlanIndMapper, as the default inbound mapper is empty.

  • A trunk port or a Provider Network Port accepts the VLANs that it carries in both filters, and needs no mapper entry.

The FiltersAndMappers configuration makes the vlanPolicy of every VLAN-aware bridge a VlanPolicyLayer:

[Config FiltersAndMappers]
description = "two service providers, each with C-VLANs for its customers, share a core that separates them by S-VLAN; the bridge ports are configured with the filters and mappers of VlanPolicyLayer"

# The VLAN policy of each bridge consists of the filters and mappers of
# VlanPolicyLayer. The filters check the VLAN id as it is on the wire.
*.sp*.bridging.vlanPolicy.typename = "VlanPolicyLayer"
*.peb*.bridging.vlanPolicy.typename = "VlanPolicyLayer"
*.pb.bridging.vlanPolicy.typename = "VlanPolicyLayer"

VlanPolicyLayer needs one more setting. By default, the direction reverser turns the PCP of a received tagged frame into a PCP request. The outbound mapper of a port that sends the VLAN untagged removes only the VLAN request, and the remaining PCP request makes the 802.1Q layer insert a tag with VLAN id 0, a priority tag. Neither the hosts nor the service provider bridges expect such tags, so PCP forwarding is turned off. (Ieee8021adPolicy removes the PCP request itself on such ports.)

# A forwarded PCP would keep the tag on a port that sends the VLAN untagged,
# as a priority tag (VLAN id 0).
*.sp*.bridging.directionReverser.reverser.forwardPcp = false
*.peb*.bridging.directionReverser.reverser.forwardPcp = false
*.pb.bridging.directionReverser.reverser.forwardPcp = false

The service provider bridges have the entries of access ports on the ports of the customer sites. The uplink to the core (eth3) is a trunk port, so the mappers have no entry for it:

# The bridges of the service providers are C-VLAN bridges. Their access ports
# accept untagged frames only and classify them into the C-VLAN of the
# customer; the uplink to the core is a trunk that carries both C-VLANs
# C-tagged. A customer switch takes one access port, two hosts take two.
*.sp*.bridging.vlanPolicy.inboundMapper.typename = "VlanIndMapper"
*.sp*.bridging.vlanPolicy.inboundFilter.acceptedVlanIds = {"eth0" : [-1], "eth1" : [-1], "eth2" : [-1], "eth3" : [10, 20]}
*.sp*.bridging.vlanPolicy.outboundFilter.acceptedVlanIds = {"eth0" : [-1], "eth1" : [-1], "eth2" : [-1], "eth3" : [10, 20]}

*.sp1w.bridging.vlanPolicy.inboundMapper.mappedVlanIds = {"eth0" : {"-1" : 10}, "eth1" : {"-1" : 10}, "eth2" : {"-1" : 20}}
*.sp1w.bridging.vlanPolicy.outboundMapper.mappedVlanIds = {"eth0" : {"10" : -1}, "eth1" : {"10" : -1}, "eth2" : {"20" : -1}}

*.sp1e.bridging.vlanPolicy.inboundMapper.mappedVlanIds = {"eth0" : {"-1" : 10}, "eth1" : {"-1" : 20}, "eth2" : {"-1" : 20}}
*.sp1e.bridging.vlanPolicy.outboundMapper.mappedVlanIds = {"eth0" : {"10" : -1}, "eth1" : {"20" : -1}, "eth2" : {"20" : -1}}

*.sp2w.bridging.vlanPolicy.inboundMapper.mappedVlanIds = {"eth0" : {"-1" : 10}, "eth1" : {"-1" : 20}, "eth2" : {"-1" : 20}}
*.sp2w.bridging.vlanPolicy.outboundMapper.mappedVlanIds = {"eth0" : {"10" : -1}, "eth1" : {"20" : -1}, "eth2" : {"20" : -1}}

*.sp2e.bridging.vlanPolicy.inboundMapper.mappedVlanIds = {"eth0" : {"-1" : 10}, "eth1" : {"-1" : 10}, "eth2" : {"-1" : 20}}
*.sp2e.bridging.vlanPolicy.outboundMapper.mappedVlanIds = {"eth0" : {"10" : -1}, "eth1" : {"10" : -1}, "eth2" : {"20" : -1}}

The edge bridges of the core have the entries of Customer Network Ports toward the service providers and of a Provider Network Port toward the core bridge, and the core bridge has those of two Provider Network Ports:

# An edge bridge of the core accepts frames without an S-tag from a service
# provider, puts all of them into the S-VLAN of that service provider, and
# removes the S-tag when it sends to it.
*.peb*.bridging.vlanPolicy.inboundMapper.typename = "VlanIndMapper"
*.peb*.bridging.vlanPolicy.inboundMapper.mappedVlanIds = {"eth0" : {"-1" : 100}, "eth1" : {"-1" : 200}}
*.peb*.bridging.vlanPolicy.inboundFilter.acceptedVlanIds = {"eth0" : [-1], "eth1" : [-1], "eth2" : [100, 200]}
*.peb*.bridging.vlanPolicy.outboundFilter.acceptedVlanIds = {"eth0" : [-1], "eth1" : [-1], "eth2" : [100, 200]}
*.peb*.bridging.vlanPolicy.outboundMapper.mappedVlanIds = {"eth0" : {"100" : -1}, "eth1" : {"200" : -1}}

# The core bridge relays by S-VLAN only.
*.pb.bridging.vlanPolicy.inboundFilter.acceptedVlanIds = {"eth0" : [100, 200], "eth1" : [100, 200]}
*.pb.bridging.vlanPolicy.outboundFilter.acceptedVlanIds = {"eth0" : [100, 200], "eth1" : [100, 200]}

In the walk of the frame through pebW above, steps 2 and 6 take place in the submodules of the vlanPolicy. In step 2, the inboundFilter sees no S-VLAN, that is, -1, which eth0 accepts, and the inboundMapper puts the frame into S-VLAN 100. In step 6, the outboundMapper has no entry for eth2, so the frame keeps S-VLAN 100, which the outboundFilter of eth2 accepts. On the way back, the outboundMapper of eth0 maps S-VLAN 100 to -1, that is, it removes the S-VLAN request. A broadcast, such as the first ARP request, is sent to all other ports by the relay, and the outbound filters drop it on the ports that are not members of its VLAN.

The two configurations differ in one point that this network does not exercise: the [-1] filter entries of the access ports and Customer Network Ports drop priority-tagged frames, which Ieee8021adPolicy admits. Accepting 0 in the inbound filter, and mapping it like -1 in the inbound mapper, would admit them too.

Results

The First Ping

hostA1a starts pinging 1 ms before the other hosts, so its first exchange crosses the network alone. The following video shows it: hostA1a first resolves the MAC address of hostA2a with ARP, then sends the ping request, and hostA2a answers:

The ARP request is a broadcast, so each bridge on its way floods it, but only to the other ports of its VLAN: sp1w leaves out its port toward customer B, pebW and pebE their ports toward service provider 2, and sp1e its ports toward customer B. The request thus reaches only the hosts of customer A. The frames on the links between pebW, pb and pebE carry both tags.

Frames on the Wire

The following images show the ping request of hostA1a on both sides of pebW, as the packet inspector of Qtenv shows the frame when it arrives at the end of the link: the list of its chunks under dissected data, each with its fields.

On eth0, toward service provider 1, the frame carries only the C-tag of customer A, with C-VLAN 10:

../../src/media/showcases/ethernet/sharedcore/doc/frame_pebW_eth0_hostA1a.png

The EthernetPhyHeader chunk is the preamble and the start frame delimiter. The typeOrLength fields are shown in decimal: 33024 in the MAC header is 0x8100, the TPID of the C-tag, and 2048 in the tag is 0x0800, IPv4. (Note that the typeOrLength field of a tag indicates the type of the next header, not its own.)

On eth2, toward the core bridge, the same frame has an S-tag with S-VLAN 100 in front of the unchanged C-tag:

../../src/media/showcases/ethernet/sharedcore/doc/frame_pebW_eth2_hostA1a.png

Here, the typeOrLength of the MAC header is 34984, that is, 0x88a8, the TPID of the S-tag, and the first tag announces the C-tag with 33024. Both tags are Ieee8021qTagEpdHeader chunks; which kind of tag a chunk is follows from the TPID in the typeOrLength field before it.

A protocol analyzer names the two tags. The PCAP file of eth2 shows the same frame in Wireshark’s terms, without the preamble:

$ tshark -r results/PortTypes-#0-pebW_eth2.pcap -V -O eth,vlan,ieee8021ad \
    -Y 'icmp.type==8 && ip.src==10.1.10.1 && icmp.seq==0'
Frame 3: 110 bytes on wire (880 bits), 110 bytes captured (880 bits) on interface eth2, id 0 (outbound)
Ethernet II, Src: 02:00:00:00:0a:1a (02:00:00:00:0a:1a), Dst: 02:00:00:00:0a:2a (02:00:00:00:0a:2a)
    Type: 802.1ad Provider Bridge (Q-in-Q) (0x88a8)
IEEE 802.1ad, ID: 100
    Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 10
    Type: IPv4 (0x0800)
    Trailer: b6086a6b
Internet Protocol Version 4, Src: 10.1.10.1, Dst: 10.1.10.3
Internet Control Message Protocol

The Trailer line is the frame check sequence, which Wireshark does not recognize as such behind the VLAN tags; it is the fcs of the EthernetFcs chunk in hexadecimal.

The ping request of hostC1a, a customer of service provider 2, crosses the same link. Its MAC addresses and its C-tag are identical to those of hostA1a’s frame; only the S-VLAN differs:

../../src/media/showcases/ethernet/sharedcore/doc/frame_pebW_eth2_hostC1a.png

Over the whole simulation, pebW’s port toward service provider 1 carries only C-tagged frames of the subnets of service provider 1. Its port toward the core bridge carries S-VLAN 100 with the subnets of service provider 1, and S-VLAN 200 with those of service provider 2, each over C-VLAN 10 and 20.

MAC Learning in the Core

At the end of the simulation, the MAC forwarding table of the core bridge pb holds 16 entries for the 8 different MAC addresses of the hosts: each address is learned once in S-VLAN 100 and once in S-VLAN 200. Interface 100 is eth0, toward pebW, and interface 101 is eth1, toward pebE:

../../src/media/showcases/ethernet/sharedcore/doc/pb_mactable.png

The two copies of each address do not interfere, because they are learned in different S-VLANs.

Ping Results

Each of the 16 hosts sent 5 ping requests and received 5 replies, so all 80 pings were answered, with 0% loss. The numbers come from the Pings sent, pingRxSeq:count and ping loss rate (%) scalars of the ping applications, which are the same for every host.

Sources: omnetpp.ini, SharedCoreShowcase.ned, addresses.xml

Try It Yourself

If you already have INET and OMNeT++ installed, start the IDE by typing omnetpp, import the INET project into the IDE, then navigate to the inet/showcases/ethernet/sharedcore folder in the Project Explorer. There, you can view and edit the showcase files, run simulations, and analyze results.

Otherwise, there is an easy way to install INET and OMNeT++ using opp_env, and run the simulation interactively. Ensure that opp_env is installed on your system, then execute:

$ opp_env run inet-4.8 --init -w inet-workspace --install --build-modes=release --chdir \
   -c 'cd inet-4.8.*/showcases/ethernet/sharedcore && inet'

This command creates an inet-workspace directory, installs the appropriate versions of INET and OMNeT++ within it, and launches the inet command in the showcase directory for interactive simulation.

Alternatively, for a more hands-on experience, you can first set up the workspace and then open an interactive shell:

$ opp_env install --init -w inet-workspace --build-modes=release inet-4.8
$ cd inet-workspace
$ opp_env shell

Inside the shell, start the IDE by typing omnetpp, import the INET project, then start exploring.

Discussion

Use this page in the GitHub issue tracker for commenting on this showcase.