Migrate Helpdesk from Cerberus 5 to Zendesk

After seven years of using Cerberus as our helpdesk system at INEX, we decided it was time to upgrade / move on. Following a fairly exhaustive (and painful!) search, we settled on Zendesk as having the right mix of features:

  • agent collision detection
  • a very nice API with an official PHP client supporting composer
  • a familiar workflow to what we were used to
  • enough configurable settings to make it do what we wanted without having to wade through a kitchen sink of options
  • mobile apps for both iOS and Android

The first hurdle was migrating all the existing tickets from Cerberus to Zendesk. There’s a number of ways we could have gone about this but the Cerberus schema was fairly intuitive and Laraval makes bootstrapping an application with database access really easy. As such, we simply iterated over the tickets in the Cerberus database and imported them into Zendesk (with attachments and by creating organisations and users on the fly).

See the code on GitHub at inex/cerb5-to-zendesk.

Doctrine2 Provider for Laravel 5 with Authentication Support

About a year ago, I released a Doctrine2 provider for Laravel 4. We’ve just updated this as a new package for Laravel 5.  Full details can be seen in the project’s README.md.

This package creates four bindings into Laravel’s IoC:

  • Doctrine\ORM\EntityManagerInterface (which is an instance of Doctrine\ORM\EntityManager), also available via the D2EM facade;
  • Doctrine\Common\Cache\Cache (which is an instance of the appropriate cache provider), also available via the D2Cache facade;
  • Doctrine\ORM\Mapping\ClassMetadataFactory (used in this package by the console generator commands); and
  • the Doctrine2Bridge\Support\Repository helper object which used by the D2R facade to instantiate repositories such as D2R::r( 'User' ).

An example of using the D2Cache and D2EM

D2Cache::save( $key, $value );
D2Cache::fetch( $key );

$object = new Entities\Object;
D2EM::persist( $object );

It includes support for Laravel’s authentication system by way of a Doctrine2 based user provider with some example classes. Simply put – this allows a Doctrine2 database table storing users’ usernames and passwords to be used as the backend for Laravel 5 authentication.

Full documentation of the authentication with examples can be found here.

Available on Packagist and forkable on GitHub.

Debugging NFS Slowness

During patching for the recent GHOST bug, I updated all packages (including kernel) on a Ubuntu 14.04 file server (filer). This filer provided static content (mainly tens of thousands of images) to a number of web servers. You can see the effect in the following load graph from the filer:

Load average on the filer
Load average on the filer

You may notice from the above, that there were actually two issues. The first was solved by upgrading the filer from 14.04 to 14.10 based on a number of online references to symptoms and fixes. About an hour after this upgrade, a new form of NFS slowness manifested and, needless to say, sites that rendered in <1sec were now taking >15secs.

Diagnosing the second issue took a while longer but some tips and utilities include:

  • check /var/log and see if any log files are increasing rapidly;
  • check top and check any processes with high / unusual utilisation;
  • use iostat (apt-get install sysstat) and pay particular attention to any devices with high volumes of transactions per second. In my case it was the root filesystem rather than any of the mounted partitions exported by NFS.
  • use iotop (apt-get install iotop) and note any processes with high utilisation (in my case jbd2/xvda1-8 was at 100% and xvda1-8 is my root partition)

The jbd2 process is the ext4 journaling process. At this point you can evaluate fsck’ing your partition but I wanted to see if I could discover what was happening here. I enabled some debugging via:

# enable tracing:
echo 1 > /sys/kernel/debug/tracing/events/ext4/ext4_sync_file_enter/enable
# wait a couple of seconds and:
cat /sys/kernel/debug/tracing/trace
# and disable tracing:
echo 0 > /sys/kernel/debug/tracing/events/ext4/ext4_sync_file_enter/enable

What I found were lots of:

nfsd-2085  [001] .... 53730942.155573: ext4_sync_file_enter: dev 202,1 ino 276278 parent 149955 datasync 0
nfsd-2071  [001] .... 53730942.158743: ext4_sync_file_enter: dev 202,1 ino 276278 parent 149955 datasync 0

where every entry related to the same inode number (276278). We found this via:

find / -inum 276278

The solution was to stop nfs_kernal_server, remove that directory entirely, add it back and restart the nfs_kernel_server. We got the permissions wrong on the first attempt but this’ll be obvious from dmesg / kernel log messages such as:

kernel: [53731827.778104] NFSD: Failed to remove expired client state directory 8d97cccceb37641d3804a84683a9282a
kernel: [53731827.779204] NFSD: failed to write recovery record (err -13); please check that /var/lib/nfs/v4recovery exists and is writeableNFSD: Failed to remove expired client state directory 8d97cccceb37641d3804a84683a9282a

Nagios Plugin to Check Extreme Networks Devices

Over at INEX we’ve embarked on a forklift upgrade of the primary peering LAN using Extreme Networks Summit x670’s and x460’s. As usual, we need to monitor these 24/7 and we have just written a new Extreme Networks chassis monitoring script which should work with most Extreme devices.

It will check and generate alerts on the following items:

  • a warning if the device was recently rebooted;
  • a warning / critical if any found temperature sensors are in a non-normal state;
  • a warning / critical if any found fans are in a non-normal state;
  • a warning / critical if any found PSUs are in a non-normal state (or missing);
  • a warning / critical if the 5 sec CPU utilisation is above set thresholds;
  • a warning / critical if the memory utilisation is above set thresholds.

You’ll find the script in this Github repository: barryo/nagios-plugins.

A some verbose output follows:

./check_chassis_extreme.php -c  -h -v 
CPU: 5sec - 4% 
Last reboot: 4007.7666666667 minutes ago 
Uptime: 2.8 days. 
PSU: 1 - presentOK (Serial: 1429W-xxxxx; Source: ac) 
PSU: 2 - presentOK (Serial: 1429W-xxxxx; Source: ac) 
Fan: 101 - OK (4388 RPM) 
Fan: 102 - OK (9273 RPM) 
Fan: 103 - OK (4428 RPM) 
Fan: 104 - OK (9273 RPM) 
Fan: 105 - OK (4551 RPM) 
Fan: 106 - OK (9452 RPM) 
Temp: 39'C 
Over temp alert: NO 
Memory used in slot 1: 29% 
OK - CPU: 5sec - 4%. Uptime: 2.8 days.  PSUs: 1 - presentOK; 
  2 - presentOK;. Overall system power state: redundant power 
  available. Fans: [101 - OK (4388 RPM)]; [102 - OK (9273 RPM)];
  [103 - OK (4428 RPM)]; [104 - OK (9273 RPM)]; [105 - OK (4551 
  RPM)]; [106 - OK (9452 RPM)];. Temp: 39'C. 
  Memory (slot:usage%): 1:29%.

OS X Built-in tftp Server

Turned out to be very useful during a recent RMA maintenance window:

sudo launchctl load -F /System/Library/LaunchDaemons/tftp.plist
sudo launchctl start com.apple.tftpd

The default tftp file path is /private/tftpboot. [Original source]

You can stop it with:

sudo launchctl unload -F /System/Library/LaunchDaemons/tftp.plist

And, speaking of tftp – there are some interesting projects on GitHub: hooktftp, php-tftpserver and ptftpd.

5 Days in Dubai

A review of our recent trip to Dubai (‘our’ being myself, my wife and our daughter Emily – 5yo). Now, I’ll be honest, Dubai was not high on my hit list of places to visit – mainly due to a roaring sun and many preconceptions (mostly proven wrong). However, my brother-in-law has been living and working there for the past year so the trip became inevitable. If you want to skip the review, I can sum it up as follows: I’d go back in a heartbeat.

A review of our recent trip to Dubai (‘our’ being myself, my wife and our daughter Emily – 5yo). Now, I’ll be honest, Dubai was not high on my hit list of places to visit – mainly due to a roaring sun and many preconceptions (mostly proven wrong). However, my brother-in-law has been living and working there for the past year so the trip became inevitable. If you want to skip the review, I can sum it up as follows: I’d go back in a heartbeat.

The flight out with Emirates was perfectly timed: departing Dublin T2 on Monday evening at 20:20 for a 7 hours 30 minutes flight and landing, with a +4 hour time difference, at 07:50 local time. I was never going to sleep much on this but Emily slept almost the entire way and Cherrie nabbed 40 winks.


Arrival into Dubai International Airport (DXB) T3 was easy. There was a Costa coffee outlet in baggage claim and immigration was painless (no visa required for Irish passport holders on a holiday). We knew taxis were cheap but, having time to kill while hoping for an early check-in, we decided to take the metro. We knew our hotel, the Conrad, was situated near World Trade Centre stop but, as it turned out, it’s right at the stop.

Emily looking down on Dubai
View from the 46th floor.

This convenience sold us on the metro for the week. We picked up two Nol cards (translates as fare cards – equivalent to Leap cards) and loaded them up with 40 dirhams each (~€8.80). We took 2 – 3 return metro trips a day from Tuesday to Saturday and didn’t need to top them up again. Great value and very convenient. But also one of the best ways to get a real sense of the people of Dubai.

Some quick facts: Dubai is both the name of the emirate and the city. It has a population of ~2m which has doubled in 10 years. Only 15% are Emirati nationals, the rest are expatriates and about 85% of those are Asian.

We arrived into the hotel at 9AM and, thankfully, they had a room available on the 46th floor – the top floor! We unpacked, washed the stench of travel off, refreshed and headed back out. The plan was to meet Cherrie’s brother Conor later that afternoon and plan the week. We decided Dubai Mall was an easy target being only four metro stops away.

Waterfall in Dubai Mall.
Waterfall in Dubai Mall.

Dubai Mall will be on everyone’s todo list – it’s situated next to the tallest building in the world – the Burj Khalifa. At this point it’s important to mention that Emily was extraordinary every day in the kilometres of walking she had to do without complaining. We pretty much metro’d and walked wherever we could. Arriving at the Dubai Mall metro stop, you have to walk 1.7Km along a huge enclosed and air conditioned gangway to reach the mall. The entrance is a geeks delight – about 10 electronic / gadget / computer stores.

The aquarium in Dubai mall.
The aquarium in Dubai mall.

It’s impossible to over exaggerate the size and scale of Dubai Mall. It is the world’s largest mall based on total area with 1,200 shops. Forget Dundrum Town Centre with its 100 and odd shops! I won’t claim we got our bearings but we had a wander around, grabbed lunch in a food court where all of the world’s cuisines were available and took in some sights such as the indoor waterfall, the built in aquarium and grabbed some essentials in the Waitrose supermarket.

Then it was back to the hotel to meet Conor (who was working nights this week) and plan an itinerary for the week. Followed by a swim in the hotel pool – a favourite of Emily’s and the perfect cool tonic to the heat.

By the evening we were crashing fast. Understandably. Conor guided us to an Irish bar with good food across the road from the hotel. Probably the last place in Dubai I’d have chosen pre-arrival but, after a long day of walking after a sleepless overnight flight, something close, convenient and familiar fit the bill in the moment. McGettigan’s served up some good burgers, Guinness but a poor fish’n’chips.


Crossing the creek on an abbra.
Crossing the creek on an abbra.

We decided to hit the Dubai City Souks around Deira (Al Ras metro stop) – we didn’t get around to the fish market unfortunately but we were enveloped in the aromas of the Spice Souk and the Perfume Souk, marvelled at the Gold Souk and dodged offers of fake handbags and watches in the Covered Souk (that was kept for later!). We then took an abbras (small wooden dhows) – water taxi – across the creek to Bur Dubai Souk.


Looking back a Deira.
Looking back at Deira.

Despite being told that you don’t walk anywhere in Dubai before leaving, this was part of the original Dubai with plenty of streets to meander around. There’s also lots of small food stalls and – if you don’t have hygiene phobias – you’ll easily pick up a chicken shawarma for about 50cent.

Back to the hotel then to meet Conor via the Al Ghubaiba metro stop.


Cruising around a lake in Safa Park.
Cruising around a lake in Safa Park.

We decided to take a walk around Safa Park with Conor – grab an ice cream and spin around a lake on a boat. It’s a 64 hectare urban park with three lakes, over 200 species of birds, and 16,924 different trees and bushes. It’s also one of the few places you’ll find grass rather than sand.

We finished up Tuesday by heading back to Dubai mall to get our dinner and to check out the famous fountainsset on the 30-acre Burj Khalifa Lake, the fountain shoots water jets as high as 150 metres, equivalent to that of a 50-storey building. The fountain is 275 metres long and has five circles of varying sizes and two central arcs. It has been designed by California-based WET, the creators of the Fountains of Bellagio in Las Vegas.

2014-10-29 20.00.37 2014-10-29 21.27.05 2014-10-29 21.27.45





We found a nice Italian but no alcohol on sale here. You can get a drink, just mainly in hotels and other licensed places.


2014-10-30 12.12.51After two days on the Northern end of the city, we decided to head south to Jumeirah via the Dubai Marina metro stop (which interestingly, for me, brought us past Dubai Internet City). There’s a lovely marina here but the more interesting walk is to head towards Jumeirah Open Beach and Walk – a couple of kilometres of a pedestrianised zone of boutique shops, cafes and beach. The heat at this time of year is not unbearable but a Starbucks Frappuccino has never tasted so good! Back to the hotel for lunch and a quick dip to cool off.

2014-10-30 16.35.01The afternoon heralded a complete change of scenery as our guide, Umar, collected us in his Land Rover jeep for a desert safari (with two others). This took us out of Dubai – which quickly changed from metropolis to desert – for about 6 hours. Our first stop was quad biking which was exhilarating – dune bashing at speed over and into sand dunes. Emily even got a spin on the quad around the flatter areas.

Little did we realise that after a bit of dune bashing on a quad, we were about to do it for real. These images look pretty tame compared to our experience. Umar was taking no prisoners and Emily – who enjoyed the first five minutes – turned quickly grey while breaking out in a cold sweat for the next ten minutes. I’d be lying if I said I didn’t sweat a bit myself as I glanced parts of cars left behind whizzing past the window!

2014-10-30 17.14.30 2014-10-30 17.17.20 2014-10-30 17.19.33 2014-10-30 17.21.48 2014-10-30 17.22.28 2014-10-30 17.00.22








After some photo ops in the dessert, we were brought to a dessert campsite for a traditional dinner, belly dancing, camel rides, henna tattoos and more.

2014-10-30 18.07.00 2014-10-30 17.57.33 2014-10-30 18.29.41






Friday (Holy Day – our Sunday)

We kicked the morning off with a water taxi from Al Ghubaiba on one end of the the city to Dubai Mall Marina on the other. We taxi’d to Al Ghubaiba when we realised the Metro doesn’t kick off until 1PM. Taxis, by the way, are also very cheap. The boat ride took about 1 hour 30 minutes and gave a fantastic view of all of Dubai from the see as well as the (incomplete) The World, the Burj Al Arab and Jumeirah Palm.

2014-10-29 13.15.16 2014-10-31 10.40.24 2014-10-31 10.44.38 2014-10-31 11.38.47 2014-10-31 11.50.42 2014-10-31 12.01.20









A quick bit of shopping in our destination – the much more manageable Dubai Mall Marina – with a spot of brunch and then the metro back to the hotel. The afternoon was spent with Conor in a suburb known for its vast quantities of high quality but fake handbags and watches. The girls wrapped up their Christmas shopping here.

2014-10-31 19.33.11Conor and his girlfriend Laura took Emily home for the night leaving myself and Cherrie to head out ‘properly’. Conor directed us to Madinat Jumeirah – a hotel resort – which reminded me a lot of the Venetian in Las Vegas. We had a lovely meal here (take your pick of many restaurants) and, importantly, a bottle of wine!

We taxi’d from there to the Burj Kahilfa where our destination was At.Mosphere on the 122 floor. A bar with some lovely looking tapas – alas my food quota had been reached. We may have thought we were high on the 46th floor of the hotel, but 122 floors up is something entirely different. And that was only just over half way up the building. There’s no entry fee but a minimum spend of 300 dirhams per person applies (~€65). This is easy to reach with about 3 – 4 drinks each. Reservations aren’t required but I’d strongly recommend it. This place is at it’s best if you can ring ahead to get a window seat looking down on the city.


wild-wadi-rides-jumeirah-sceirah-final-heroConor, Laura and Emily collected us Saturday morning to head to Wild Wadi Waterpark. Conor’s residency card got us a good 40% discount on the expensive headline rates but it’s well worth the cost – even if you’re just a party of adults. We spent a good few hours here enjoying the slow streams to being scared witless on the Jumeirah Sceirah.

One question was also answered here – what do women who normally wear a Hijab do in a place like Wild Wadi? Well, they wear a burqini. A swimsuit equivalent of the Hijab. Dubai is very much a melting pot of cultures and we had no issues anywhere we went in terms of culture or clothing. Cherrie, Laura and many other non-Islamic girls in their bikinis co-mingled without issue / stares / comment with Islamic women in their burqini’s. There just was no issue here. I, however, will confess to being somewhat jealous of the burqini from a purely sun protection point of view!

The evening was spent with a return trip to Dubai mall for a last supper with Conor and Laura and a final spot of shopping.

2014-11-01 18.50.28 2014-11-01 19.50.12 2014-11-01 19.50.39





Then early to bed for a 4AM rise to get to the airport for the trip home.

Notes / Comments / Takeaways

  • We all loved Dubai and would happily go back. We wouldn’t want to live our lives there but would happily spend a few years working there (under a different life path!).
  • Forget whatever you’ve been told about dress code / covering up. We saw all fashion types form skimpy to Nijab co-exist in all places we visited. Don’t assume that that extends outside Dubai though.
  • Also forget the rumours of the UAE’s big firewall blocking access to (or download of) social media apps. One of their biggest mobile companies were running an offer of unlimited social data for x dirham a month where two of the four sample apps were Facebook and Twitter. Both of which seem to be in widespread usage there.
  • If you dash for the metro and, after catching your breath, look around and notice a lot of women looking at you with a smirk on their faces with no other men in sight, you’re most likely in the women and children only carriage. You won’t be arrested for this! And don’t worry, you’ll find women and children in all carriages, this is just for those who prefer to travel without men under whatever traditions / beliefs they have.
  • We booked the flights and hotel as a package via Expedia. We stayed in the Conrad Dubai but, with the way we travelled, any hotel near a metro would have been fine.