Peering Week Articles on trefor.net

I spent the first few days of St Patrick’s week last month in Leeds at the first of the two annual Euro-IX conferences on behalf of INEX. Trefor Davies, of trefor.net, organised a series of articles called Peering Week on his blog to coincide with it:

During Peering Week we have had 18 excellent contributions from some of the people who run the internet in Europe. This might sound dramatic especially considering that the internet is made up of sixty or seventy thousand Autonomous Networks. The contributors this week run Internet Exchanges where a greats many of these networks connect to each other.

My contribution was about our IXP management system called IXP Manager – co-written by myself and Nick Hilliard for INEX. This tool is now being used to manage two IXPs in the UK, at least five more across Europe, a couple that we know about in the US and it is now the de facto choice for IXPs in Africa and Asia – where we are working with ISOC.

You can read the full article on Tref’s blog here: INEX’s IXP Manager – tools to help manage an Internet Exchange.

I’m glad to say that the good folks at Euro-IX helped ensure I wasn’t too homesick on St. Patricks’s Day – as the days proceedings wrapped up, we were greeted by:

guinness_array_header

Bird / Quagga with MD5 Support for IPv4/6 on FreeBSD & Linux

Over in INEX we run a route server cluster which alleviates the burden of setting up bilateral peering sessions for the more than 80% of the members that use them. The current hardware is now about six years old and we have a forklift upgrade in the works.

BGP allows for MD5 authentication between clients (using the TCP MD5 signature option, see RFC 2385) and – while recently obsoleted in RFC 5925 – it is still widely used in shared LAN mediums such as IXPs; primarily to prevent packet spoofing and session hijacking via recycled IP addresses.

Our current route server implementation runs on FreeBSD which does not support TCP MD5 in its stock kernel (you are required to compile a custom kernel – see below for details). Additionally, specifying the session MD5 is not done in the BGP daemon configuration but separately in the IPsec configuration. Lastly, our current FreeBSD version has no support for TCP MD5  over IPv6. These have all led to unnecessarily complex configurations and a degree of confusion.

Because of this, we decided to test up to date Linux and FreeBSD versions for native IPv4 and IPv6 TCP MD5 support with Bird and Quagga (our route server daemons of choice).

In each case, BGP sessions were tested for:

  • no MD5 on each end (expected to work);
  • same MD5 on each end (expected to work);
  • different MD5 on each end (expected not to work); and
  • MD5 on one end with no MD5 on the other end (expected not to work).

For Linux, the platform chosen was Ubuntu 12.04 LTS with the stock 3.2.0-40-generic kernel.

  • Sessions were tested for Quagga to Quagga and Quagga to Bird;
  • Sessions were tested over both IPv4 and IPv6;
  • The presence of valid MD5 signatures were confirmed using tcpdump -M xxx;
  • Stock Quagga and Bird from the 12.04 apt repositories were used.

The results – everything worked and worked as expected:

  • BGP sessions only established when expected (no MD5 configured, same MD5 configured);
  • This held for both IPv4 and IPv6.

Summary: Linux will support TCP MD5 nativily for IPv4 and IPv6 when using Quagga or Bird.

For FreeBSD, we used the latest production release of 9.1. TCP MD5 support is not compiled in by default so a custom kernel must be built with the additional options of:

In addition to this, the MD5 shared secrets need to be added to the IPsec SA/SD database via the setkey utility or, preferably, via the /etc/ipsec.conf file which, for example, would contain entries for IPv4 and IPv6 addresses such as:

where the addresses ending in .1/:1 are local and .2/:2 are the BGP neighbor addresses. This file can be processed by setting ipsec_enable="YES" in /etc/rc.conf and executing /etc/rc.d/ipsec reload.

  • Sessions were tested for Quagga/Linux to Quagga/FreeBSD and  from Quagga/Linux to Bird/FreeBSD;
  • Sessions were tested over both IPv4 and IPv6;
  • The presence of valid MD5 signatures were confirmed using tcpdump -M xxx;
  • Stock Quagga from the 12.04 apt repositories and stock Quagga and Bird from FreeBSD ports were used.

The results – almost everything worked and worked as expected:

  • BGP sessions only established when expected (no MD5 configured, same MD5 configured);
  • This held for both IPv4 and IPv6;
  • one odd but expected behavior – you only need to set the MD5 via setkey / ipsec.conf – setting it (or not) in the Quagga and Bird config has no effect so long as it is set via setkey (but is useful for documentation purposes). However, trying to set it in Quagga without having rebuilt the kernel will result in an error.

Summary: FreeBSD will support TCP MD5 via a custom kernel and setkey / ipsec.conf for IPv4 and IPv6. Note that there is an additional complexity when changing or removing MD5 passwords as these need to be amended / deleted via setkey which can put an extra burden on automatic route server configuration generators.

What the Hell is INEX? An IXP?

In a few recent posts, I’ve mentioned INEX.

INEX is a neutral, industry-owned association, founded in 1996, that provides IP peering facilities for its members. INEX membership is open to all organisations that can benefit from peering their IP traffic and there are currently 57 members.

INEX can also be considered Ireland’s IP Peering Hub. INEX membership provides high-speed, reliable and resilient IP traffic exchange facilities for both Irish and International organisations, allowing them to route IP traffic efficiently thereby providing faster, more reliable and lower-latency internet access for their customers.

So what the hell is an IXP? Well, Euro-IX commissioned the following, the Internet Revealed: A file about IXPs, a couple of years ago which brilliantly explains IXPs.

Follow Up – IPv6 Statistics at INEX

A couple of days ago, I was talking about World IPv6 day with some notes on the Irish context.

INEX is a neutral, industry-owned association, founded in 1996, that provides IP peering facilities for its members. INEX membership is open to all organisations that can benefit from peering their IP traffic and there are currently 57 members.

INEX can also be considered Ireland’s IP Peering Hub. INEX membership provides high-speed, reliable and resilient IP traffic exchange facilities for both Irish and International organisations, allowing them to route IP traffic efficiently thereby providing faster, more reliable and lower-latency internet access for their customers.

As a follow up to the previous post, here’s a like for like comparison of IPv4 and IPv6 traffic over peering LAN 1 of the exchange:

Notes:

  • As a layer 2 exchange, traffic over INEX is symmetrical – traffic originating from one member is destined for another.
  • INEX runs two peering LANs for resiliency. The IPv6 traffic on LAN 2 was negligible over the same period. See the public statistics and the weathermaps of each LAN showing the network topology.

 

World IPv6 Day with Irish Statistics

In case it passed you by, today was World IPv6 Day. In a nutshell: “Major Internet service providers (ISPs), home networking equipment manufacturers, and web companies around the world are coming together to permanently enable IPv6 for their products and services by 6 June 2012.” This includes top content providers such as Facebook (see under their hood), Google (read what they had to say), Yahoo! and Microsoft. In fact, you may not even have noticed but Google were advertising it front and centre on their search page:

Google Announcing World IPv6 Day on Their Search Page

Over at INEX, we were unable to pull out IPv6 traffic statistics on the exchange until recently and my colleague just got the first pass of that project complete this week in time for World IPv6 Day. Here’s how it looked over the hours leading up to and into World IPv6 Day:

Now, the peek of almost 40Mbps is, most assuredly, small compared to the overall peek of 24Gbps, but there is a very pronounced jump in IPv6 traffic which is certainly a good sign and a move in the right direction. The overall peering statistics at INEX are public and we’ll be breaking out IPv4 and IPv6 into separate graphs shortly also.

Why does IPv6 amount to < 0.2% of the traffic at the exchange? Well there are two main factors:

  • Until today, there has been very little mass or popular content available over IPv6. So, even if you were IPv6 enabled, there was very little for you access.
  • None of the large ISPs in Ireland are providing IPv6 connectivity to end users outside of certain closed test programs.

This is the classic chicken and egg problem: with no content available the ISPs were not motivated to provide IPv6 connectivity; and, conversely, with no IPv6 enabled eyeballs the content providers were not motivated to make their services available over IPv6.

While today was not necessarily a content provider only day, I’m unaware of any Irish ISPs that got involved. But, now that we have significant content available over IPv6, hopefully the ISPs will begin to ramp up their own programs. And – to be fair – it’s not all bad news with the ISPs in Ireland. Most have their core and edge networks IPv6 enabled, it’s the access layer that’s the issue (and it’s a really really big issue and a very difficult issue).

AMS-IX (the Amsterdam Internet Exchange) is in the top three IXPs in the world by traffic volume and they also make their IPv6 statistics public. As a second demonstration of traffic levels on World IPv6 Day, here is the week to date showing a huge differential for today:

If you’re not sure what all this is about, well then here are a few words from the creator of the Internet himself:

And if you’re keen to start experimenting with IPv6, first email and ask your ISP. They’ll say no, but do it anyway! Then head over to SixXS (and be sure to choose either HEAnet or Digiweb as your PoP as both are INEX members and as such you’ll have the lowest possible latency).