See https://github.com/ipspace/netsim-tools/issues/171

I propose the following:

defaults:
 ip_mtu: 9000

(and then we update all device templates to apply this consistently)

Do we need separate ipv4/ipv6 values?

When I started developing netsim-tools I had absolutely no idea we would get as far as we got. It was meant to be a collection of tools that would help you build networking labs, not a whole-blown system with an abstraction layer supporting over a dozen protocols... so the name netsim-tools seems less appropriate than it did in the past.

As the CLI interface already uses netlab command to do anything and everything (as opposed to the good old days when you had to run individual Ansible playbooks), it might make sense to rename the whole thing (including Python package) to netlab.

Thoughts?

/cc @ssasso @jbemmel @petercrocker (as I can't assign you to this issue for whatever weird reason)

A quick glance at the recent code changes shows a new 'localas_ibgp' neighbor type. It is currently only referenced within the bgp module, but it can be present in the input for templates.

This breaks logic like https://github.com/ipspace/netlab/blob/dev/netsim/ansible/templates/bgp/eos.macro.j2#L11 which assumes there are only 'ibgp' or 'ebgp' values.

This may be a work in progress - just documenting what I noticed

For some reason jinja interprets the RD as possibly a sexagesimal number? TIL: https://en.wikipedia.org/wiki/Sexagesimal

The effect is when ansible reads the host_vars file value rd: 65000:1 it interprets it as 3900001.

TASK [eos_config: deploying initial from templates/initial/eos.j2] **************************************************************************************
fatal: [h1xgw1]: FAILED! => changed=false
  data: |-
    rd 3900001
    % Ambiguous command
    localhost(config-s-ansibl-vrf-blue)#
  msg: |-
    rd 3900001
    % Ambiguous command
    localhost(config-s-ansibl-vrf-blue)#

Based on recent fixes and ongoing discussions (cc @jbemmel @ssasso) it's evident we need to redesign the IPv4/IPv6 addressing system. The solutions has to support at least:

• Single-stack IPv4, single-stack IPv6, and dual-stack setup
• Address pools, static link addressing, and static node addressing
• Unnumbered IPv4 and IPv6 LLA (per-protocol unnumbered)
• Unaddressed (l2only) links

Ideally, we'd have an easy migration path and retain (temporary) support for existing unnumbered attribute.

When attempting to implement the Multipath example using SROS/SRLinux:

[email protected]:~/srlinux/netsim-examples/BGP/Multipath$ netlab create -d srlinux -p clab -s nodes.rr.device=sros -s nodes.m.device=sros
Traceback (most recent call last):
  File "/home/jeroen/srlinux/netsim-tools/netlab", line 8, in <module>
    netsim.cli.lab_commands()
  File "/home/jeroen/srlinux/netsim-tools/netsim/cli/__init__.py", line 76, in lab_commands
    mod.run(sys.argv[arg_start:])   # type: ignore
  File "/home/jeroen/srlinux/netsim-tools/netsim/cli/create.py", line 67, in run
    read_topology.add_cli_args(topology,args)
  File "/home/jeroen/srlinux/netsim-tools/netsim/read_topology.py", line 98, in add_cli_args
    common.set_dots(topo,k.split('.'),v)
  File "/home/jeroen/srlinux/netsim-tools/netsim/common.py", line 112, in set_dots
    set_dots(b[k_list[0]],k_list[1:],v)
  File "/home/jeroen/srlinux/netsim-tools/netsim/common.py", line 112, in set_dots
    set_dots(b[k_list[0]],k_list[1:],v)
  File "/home/jeroen/srlinux/netsim-tools/netsim/common.py", line 108, in set_dots
    b[k_list[0]] = v
TypeError: 'NoneType' object does not support item assignment

Some use cases require links that connect 2 ports of the same node. Due to the representation as a dict, these cannot currently be represented (type "p2p" results in an error)

Proposal: new link type ("loopback" or "s-hook")

Starting the feature design discussion with my proposal.

Some devices used on netsim may need to support VLANS / switchports on links ("L3 switch behaviour"). Some others may only need to support sub-interface tagging ("router behaviour").

Generically speaking, a "switch node" may have the following attributes:

nodes:
  switch_node:
    vlans:
    - id: 5
      prefix: 1.2.3.0/24

(the prefix key will trigger an IP assignment to the SVI)

At the same time, the link needs to have some additional attributes:

links:
- switch_node:
    switchport:
      mode: access
      native_vlan: 5
  host_node:
- switch_node:
    switchport:
      mode: trunk
      vlans: [ 5 ]
      native_vlan: 1
  router_node:
    dot1q:
      vlan: 5

Release 1.1 should go out in the next few days, potentially over the next weekend. Can't talk about plugins until we have a release that uses the new data structures, but I also want to work some more on the current code hoping to trip a bug or two.

Here's the current to-do list:

• Have to add a check of unnumbered capabilities (#107)
• Would like to add #123 to stress-test configuration templates (in which case it would be great if @ssasso and @jbemmel could do a few tests once I'm done)
• #84 would be nice to have (cc @petercrocker), but maybe it's worth waiting for #96
• It would be great if @jbemmel could add #112 and #113 for SR Linux and SR OS
• Increase code coverage (we added tons of new code)
• Documentation

Anything else you'd like to see in 1.1?

Planned for 1.1.1:

• Provider-specific device properties (#96)
• Subnet object (#103)
• MPLS module (#60)

I will probably have to take a short break before I start working on 1.1.1 and do some leisurely reading. Fast Reroute and variants comes to mind ;)

We can't use the official 1.3-epa2 image from Docker Hub because it's outdated and doesn't work correctly with virtual network devices [1]. The thing is critical for the container environment with veth devices. Hence for now we can use the unofficial image [2] that is based on VyOS nightly ISO.

1. https://github.com/vyos/vyos-1x/commit/8cf5a4f023c5459cad4c84e93f73a9ddd69be81a
2. https://github.com/sysoleg/vyos-container

Assume a topology similar to this one:

vrfs:
  common:
    import: [ red, blue, common ]
  blue:
    import: [ blue, common ]
  red:
    import: [ red, common ]

nodes:
  core1:
    module: [ bgp, ospf, vrf ]
    bgp.as: 65000
  aggr1:
    module: [ ospf ]
  aggr2:
    module: [ bgp ]
    bgp.as: 65111
    bgp.originate: [ 100.64.0.0/24 ]
  host1:
    module: []
    device: linux
  host2:
    module: []
    device: linux
  server:
    module: []
    device: linux

links:
- core1: { vrf: red }
  aggr1:
- core1: { vrf: blue }
  aggr2:
- aggr1:
  host1:
  ospf.cost: 22
- aggr2:
  host2:
- core1: { vrf: common }
  server:

Where, on core1, all the interfaces are inside a VRF.

The generated host_var file for Ansible does not have a ospf "root object", which leads to this hard failure:

    The task includes an option with an undefined variable. The error was: 'ospf' is undefined

due to the fact that almost all ospf templates have sth like:

{% if ospf.af.ipv4 is defined %}

see i.e.,

• https://github.com/ipspace/netlab/blob/dev/netsim/ansible/templates/ospf/eos.j2#L2
• https://github.com/ipspace/netlab/blob/dev/netsim/ansible/templates/ospf/cumulus.j2#L1
• https://github.com/ipspace/netlab/blob/dev/netsim/ansible/templates/ospf/ios.j2#L2

I think we should always add the root ospf: object, when the ospf module is loaded.

I like specifying links in the shorthand format (a-b instead of the full dictionary). That doesn't work well for VLAN and VRF links as they need additional attributes.

We could solve that by adding links list to global VRFs and VLANs, for example:

groups:
  routers:
    module: [ vrf ]
    members: [ r ]

nodes: [ r, h1, h2, h3, h4 ]

vrf:
  red:
    links: [ h1-r, h2-r ]
  blue:
    links: [ h3-r, h4-r ]

VRF and VLAN links would

• be transformed into the usual dictionary format
• get vrf: X or vlan.access: X attribute
• be appended to the global links list

Feedback appreciated ;)

Topology-defaults file is becoming way too large (and will get much bigger when we implement #700). We should split it into numerous smaller files and combine them either at package publishing time or at runtime (depending on how time-consuming the whole thing is).

Implementation is "trivial":

• Topology defaults can have include attribute anywhere
• include attribute specifies a glob relative to the topology-defaults.yml file
• All files matched by the glob are read and inserted as a single key-value pair into the defaults at the level of the include directive. The key is the name of the file (minus the file type), the value is whatever was read from the file.

For example:

include: modules/*.yml
devices:
  include: devices/*.yml
providers:
  include: providers/*.yml

Caveats:

• This process would work only for package topology-defaults and would not be documented outside of developer documentation
• Included files cannot have further include attributes. We have to stop somewhere, plus this nicely stops infinite recursion ;)

Packaging optimization:

• If needed, we'd have a script that would generate fully-evaluated topology-defaults.yml (or even a JSON file -- JSON is much faster to read) to be included in published Python packages.

Right now we have a generic framework that checks the validity of attribute names, the values of attributes are checked throughout the code. We could get rid of most of the value checks if we describe attributes better.

Right now, the valid global/node/vlan/vrf/module... attributes are specified as lists of valid attribute names, for example:

bgp:
  attributes:
    global: [
      af, as, next_hop_self, rr_cluster_id, rr_list, ebgp_role, as_list, sessions, activate,
      advertise_loopback, advertise_roles, community, replace_global_as ]
    node: [
      af, as, next_hop_self, rr, rr_cluster_id, originate, advertise_loopback, sessions, activate,
      community, router_id, local_as, replace_global_as ]
    vrf: [ router_id ]
    node_copy: [ local_as, replace_global_as ]
    link: [ advertise ]
    interface: [ local_as, replace_global_as ]

Instead of using attribute lists, we could use dictionaries and (optionally) describe attributes with sub-dictionaries, for example (ignoring the question of whether we're supporting 4-byte ASN or not):

bgp:
  attributes:
    vrf: 
      router_id: { type: 'ipv4' }
    link:
      advertise: { type: 'bool' }
    interface:
      local_as: { type: 'int', min_value: 1, max_value: 65535, value_error: 'BGP AS number should be between 1 and 65535' }

We would need some shortcuts, like user-defined type (doesn't make sense to repeat 'what is AS' everywhere an AS is used) and simple inheritance (local_as can appear on node or interface).

Migration process: attributes specified in the new format would go through the thorough value tests, attributes specified as lists would get their names checked. We could thus gradually migrate individual attributes or modules to the new framework.

Wouldn't it be nice if you could run a few nodes in VMs and a few others in containers? That's what I'll try to do with multi-provider topologies. Obviously we'll be limited in what can be done. I can see these easy combos:

• libvirt + containerlab
• anything + external (happily ignoring connectivity issues)

Configuration would be extremely easy: add provider attribute to nodes to specify which provider a node should use. The main provider (top-level topology attribute) would have a list of allowed secondary providers so we'll stop the unsupported combinations.

Using external provider with another one is a piece of cake (conceptually, still needs to be done): call the proper netlab up/down hooks at the proper time and move on.

Combining libvirt and clab providers will be more fun.

netlab create (or equivalent) will have to:

• Set link type to bridge for any mixed-provider segment - libvirt provider hook
• Create Vagrantfile
• Modify the lab topology (remove libvirt nodes, clean up links...) - libvirt provider hook
• Create prototype clab.yml - potentially from within the same libvirt provider hook

netlab up will have to:

• Execute vagrant up
• Collect Linux bridge names from libvirt bridge names and modify the prototype clab.yml file
• Start the containerlab part of the lab (potentially just execute containerlab deploy command)

netlab down will have to:

• Execute containerlab destroy command to bring down the containers
• Execute vagrant destroy -f to bring down the VMs
• Execute cleanup

Time to implement the discussion we had in #151:

• A new top-level attribute components would allow you to define reusable lab components.
• Each component would have links and nodes plus any valid top-level module attribute
• Links within a component could refer to nodes within the component or to global nodes (not recommended)
• Nodes within the component would have the usual nodes structure

Components would be used within nodes dictionary. The key (node name) would be the prefix used to include the component (see below), the value would have to have an include attribute that would refer to a valid component. The value could also include any valid node-level attribute.

Including a component would include all component nodes and links into the main nodes dictionary. Node names and references in links would be prefixed with the included component prefix. You could then use the prefixed names in top-level links to connect a component to the outside world.

Example: two leaf-and-spine clusters connected with a superspine. IBGP+OSPF within each cluster, EBGP with superspine.

defaults.device: frr

components:
   pod:
      nodes: [ l1, l2, s1, s2 ]
      module: [ ospf, bgp ]
      bgp.as: 65000
      links: [ l1-s1, l1-s2, l2-s1, l2-s2 ]

nodes:
  c1:
    bgp.as: 65100
  c2:
    bgp.as: 65100
  pod_1:
    include: pod
    bgp.as: 65101
  pod_2:
    include: pod
    bgp.as: 65102

links:
- c1-pod_1_s1
- c1-pod_2_s1
- c2-pod_1_s2
- c2-pod_2_s2