What the Hell is INEX? An IXP?

In a few recent posts, I’ve mentioned INEX.

INEX is a neutral, industry-owned association, founded in 1996, that provides IP peering facilities for its members. INEX membership is open to all organisations that can benefit from peering their IP traffic and there are currently 57 members.

INEX can also be considered Ireland’s IP Peering Hub. INEX membership provides high-speed, reliable and resilient IP traffic exchange facilities for both Irish and International organisations, allowing them to route IP traffic efficiently thereby providing faster, more reliable and lower-latency internet access for their customers.

So what the hell is an IXP? Well, Euro-IX commissioned the following, the Internet Revealed: A file about IXPs, a couple of years ago which brilliantly explains IXPs.

The Tor Project – Anonymity Online

For anyone with even a passing interest in the Tor Project and network, a recent presentation by Jacob Appelbaum at the TNC2012 is essential viewing. It can be viewed here (skip to about 48 minutes in).

I have of course been aware of the Tor Project for years but my opinion has always been blinkered by the notion that it’s fertile ground and enabler for spammers, illegal activity and, scariest of all, child paedophilia.

This presentation changed my mind – especially in the context (and graphs shown) in relation to recent spring risings in Egypt, Syria and other places. One exchange that stood out for me was when a gentleman (from “the fascist organisation known as” Cisco – seriously, know when to STFU) accused Jacob of being the equivalent to an arms dealer because his software is used in the Egyptian revolution but it is also supporting “abuse and bad things”. Some of Jabob’s response included (e. & o.e.):

We’re not arms dealers. I’m sorry to say it, because giving away information to all people freely and not charging for it – literally not charging for it – that’s significantly different to selling specific pieces of hardware to hunt down Falun Gong and kill them in a genocide.

There’s a huge difference there and if you don’t see the difference I would encourage you to consider what it would be like if you were that Falun Gong person.

There is a difference between that and building an anominity system where all people have access for free, to be able to read freely and to be able to speak freely.

And it is true that there is an unprecedented scale and, just like Gutenberg, it is unprecedented in an equal way that is, I think, a liberation technology if you will. I’m happy that people in Egypt used it because it’s a non-violent direct action that allows people to protect themselves.

Like if you invented penicillin for example. It is true that terrorists can heal themselves with penicillin. But I think that we would generally all agree that it is better that humanity knows how to cure disease than not to cure disease.

And it is true that bad people will always be able to do bad stuff, but the question is will good people be able to do good things? And is the intention (of Tor) for good people and regular people to be able to do good things?

I think the important thing here is that the good outweighs the bad. It is a neutral technology in the sense that everyone has anonymity – but it is not neutral in one important way which is that everybody in this room has privacy and anonymity and the ability to resist censorship in a way where previously the power dichotomy was flipped. There’s an asymmetric power relationship and now it’s the other way. And so, for example, if you can’t target someone for exploitation anymore, you’re in a totally different camp. You’re reduced to a different way of behaving and that isn’t equal but it’s not equal in the sense that it empowers every person on the planet to have that which is very very different in a positive sense from some of the things we’ve seen in the past with technology. Because the technology that wires us all together may very well be, for some of us, the very thing that strings us up.

I guess the real question and blocker for me establishing a Tor relay is the position of the Irish Gardaí and legal system and my own potential personal culpability of what may be accessed through that relay. Does anyone have a definitive answer to this?

There are a (small) number of Tor relays in Ireland however:

And quite a few users:

Tor Users in Ireland

To finish, one of the things that surprised me a lot was some of the sponsors of the Tor Project which include: the Broadcasting Board of Governors, the Naval Research Laboratory, Google, Human Rights Watch, Electronic Frontier Foundation, DARPA, the National Science Foundation and a number of educational institutions, ISPs and others.

HTTP Streaming with Encryption under Linux

For a customer of ours, we need to mass encode thousands of video files and also segment and encrypt them for use with Apple’s HTTP Streaming.

For a customer of ours, we need to mass encode thousands of video files and also segment and encrypt them for use with Apple’s HTTP Streaming. (using Amazon EC2 instances for the leg work).

On his blog, Carson McDonald, has put together a good over view of how HTTP Streaming can work under Linux a long with a segmenter.

The one piece of the jigsaw we were missing was encryption and after some work ourselves and with the help of a stackoverflow question, we have a working sequence of commands to successfully and compatibly encrypt segments for playback on Safari and other supported HTTP streaming clients:

  1. Create a key file:
    openssl rand 16 > static.key
  2. Convert the key into hex:
    key_as_hex=$(cat static.key | hexdump -e '16/1 "%02x"')
  3. At this point, let’s assume we have segmented a file of 30 seconds called video_low.ts into ten 3 second segments called video_low_X.ts where X is an integer from 1 to 10. We can then encrypt these as follows:
    for i in {0..9}; do
        init_vector=`printf '%032x' $i`
        openssl aes-128-cbc -e -in video_low_$(($i+1)).ts     
            -out video_low_enc_$(($i+1)).ts -p -nosalt        
            -iv $init_vector -K $key_as_hex

With a matching m3u8 file such as the following, the above worked fine:


What caught us out was the initialisation vector with is described in the draft IETF document as follows:

128-bit AES requires the same 16-octet Initialization Vector (IV) to
be supplied when encrypting and decrypting. Varying this IV
increases the strength of the cipher.

If the EXT-X-KEY tag has the IV attribute, implementations MUST
use the attribute value as the IV when encrypting or decrypting
with that key. The value MUST be interpreted as a 128-bit
hexadecimal number and MUST be prefixed with 0x or 0X.

If the EXT-X-KEY tag does not have the IV attribute,
implementations MUST use the sequence number of the media
file as the IV when encrypting or decrypting that media file.
The big-endian binary representation of the sequence number
SHALL be placed in a 16-octet buffer and padded (on the left)
with zeros.

Encoding Video for the HTC Desire

A useful script to encode all files passed as parameters(s) for viewing on a HTC Desire.

While I’m writing about video encoding, another task I did recently was encode a load of video files for my HTC Desire (a handset I’d strongly recommend for anyone). The main reason being that I like to watch something while pounding the threadmill in the gym.

A useful script to encode all files passed as parameters(s) (must all end in .avi) is:

#! /bin/bash


echo -en "Encoding $src\t\t\tPASS1"

ffmpeg -b 600kb -i "$src" -v 0 -pass 1 -passlogfile FF -vb 600Kb \
    -r 25 -an -threads 2 -y "$dst" /dev/null

echo -e "\tPASS2"

ffmpeg -b 600kb -i "$src" -v 0 -pass 2 -passlogfile FF -vb 600Kb \
    -r 25 -threads 2 -y -vol 1536 "$dst" /dev/null

rm FF-0.log

Encoding Full HD as FLV (for Gallery3)

I have a full HD camcorder and I wanted to stick some good quality video on my gallery for relatives to view. So, I needed to convert my sample 100MB MP4 full HD file to a suitably sized FLV for the Gallery. Here’s what I did…

I have a very nice Samsung R10 Full HD Camcorder which I bought last year. After a recent family holiday, I wanted to stick some good quality video on my gallery for relatives to view. The gallery is RC2 of the excellent Gallery 3 package which uses another excellent open source tool called Flow Player to play movies.

So, I needed to convert my test 100MB MP4 full HD file to a suitably sized FLV for the Gallery. My initial attempts with ffmpeg worked fine but the quality (sample) was very poor and changing the bit rate in different ways seemed to make no difference:

ffmpeg -i HDV_0056.MP4 -vb 600k -s vga -ar 22050 -y Test.flv
ffmpeg -i HDV_0056.MP4 -b 600k -s vga -ar 22050 -y Test.flv
ffmpeg -i HDV_0056.MP4 -vb 600k -s vga -ar 22050 -y Test.flv

I then turned to x264 and broke the process down to a number of stages:

  1. Extract the raw video to YUV4MPEG (this creates a 7GB file from my 100MB MP4):
    ffmpeg -i HDV_0056.MP4 HDV_0056.y4m
  2. Encode the video component to H.264/FLV at the specified bit rate with good quality:
    x264 --pass 1 --preset veryslow --threads 0 --bitrate 4000 \
            -o HDV_0056.flv HDV_0056.y4m
    x264 --pass 2 --preset veryslow --threads 0 --bitrate 4000 \
            -o HDV_0056.flv HDV_0056.y4m

    Note that I’m using the veryslow preset which is… very slow! You can use other presets as explained in the x264 man page.

  3. Extract and convert the audio component to MP3 (the sample rate is important):
    ffmpeg -i HDV_0056.MP4 -vn -ar 22050 HDV_0056.mp3
  4. Merge the converted audio and video back together:
    ffmpeg -i HDV_0056.flv -i HDV_0056.mp3 -acodec copy \
            -vcodec copy -y FullSizeVideo.flv

    This yields a near perfect encoding at 22MB. It’s still full size though (HD at 1920×1080).

  5. The last step is to then use ffmpeg to resize the video and it now seems to respect bit rate parameters:
    ffmpeg -i FullSizeVideo.flv -s vga -b 2000k \
            -vb 2000k SmallSizeVideo.flv

The resultant video can be seen here.

Robert Swain has a useful guide for ffmpeg x264 encoding.