Tag: links

Synchronising Microsoft Exchange to Another IMAP Server

This post is much less of a detailed how-to but rather some useful links. We were tasked with the job of sync’ing about 1,000 MS Exchange mailboxes to a Dovecot server. This needed to be done via an administrator account on the Exchange end as individual user passwords were not available.

The tool of choice for this is imapsync.  Unfortunately, there is not a single formula that will work for all as it can depend on the Exchange configuration and version as well as the use of domains on the Exchange and ActiveDirectory servers.

To help understand the various combinations of logins for Exchange, I found the following invaluable: Understanding login strings with POP3/IMAP.

Also invaluable is the imapsync FAQ – just search for mentions of Exchange.

In the end, the following worked for me (but your mileage will most definitely vary!):

./imapsync --host1 exchange-server 
    --user1 'domain/adminuser/user' --password1 'admin-password' 
    --authmech1 LOGIN 
    --host2 dovecot-server --user2 user@example.com 
    --password2 userpassword

One key element here is that when logging into Exchange as an individual user I had to use --authmech1 NTLM but if you use this auth method with the above user string, you will always end up logging into the admin’s mailbox, not the user’s. That, at least, was my experience.

MySQL 5.6 – Memcached / NoSQL Support and More

MySQL 5.6 has been released with some interesting new features and performance increases:

  • What’s New in MySQL 5.6
  • DBA and Developer Guide to MySQL 5.6
  • InnoDB Integration with memcached:MySQL 5.6 includes a NoSQL interface, using an integrated memcached daemon that can automatically store data and retrieve it from InnoDB tables, turning the MySQL server into a fast “key-value store” for single-row insert, update, or delete operations. You can still also access the same tables through SQL for convenience, complex queries, bulk operations, application compatibility, and other strengths of traditional database software.

    With this NoSQL interface, you use the familiar memcached API to speed up database operations, letting InnoDB handle memory caching using its buffer pool mechanism. Data modified through memcached operations such as ADD, SET, INCR are stored to disk, using the familiar InnoDB mechanisms such as change buffering, the doublewrite buffer, and crash recovery. The combination of memcached simplicity and InnoDB durability provides users with the best of both worlds.

  • Multi-threaded Slaves
  • Improved IPv6 Support – both in the bind to address option and the INET_ATON() function.
  • Replication improvements.

All in all, some nice new features. Especially the memcached integration.

That said, MariaDB seems to be making inroads on MySQL with some distributions considering a switch. Some interesting reading from that project includes:

Woot! We hit 50Gbps at INEX!

 

Adventures with LDAP (OpenLDAP) – SSL, Multi-Master Replication and Monitoring

In my career to date, I successfully managed to avoid all but the periphery engagement in OpenLDAP. Until recently that is – we had to build a Microsoft Exchange like environment with open source software in a way that was closely integrated and easily managed. But, more on that another time. For anyone else diving into OpenLDAP, here are some articles on my experiences that I have penned:

Nagios Plugin for Checking Backups via rsnapshot

We’ve just added a check_rsnapshot.php script to our nagios-plugins bundle on Github. This script will verify rsnapshot backups via Nagios using a number of checks / tests:

  • minfiles – checks the number of files in a snapshot against a minimum expected number;
  • minsize – checks the size of a snapshot against a minimum expected size;
  • log – parses the rsnapshot log to ensure the most recent runs for each retention period completed successfully;
  • timestamp – checks for files created server side containing a timestamp and thus ensuring snapshots are succeeding;
  • rotation – checks that retention directories are being rotated; and
  • dir-creation – checks that retention directories are being created.

Please see this Github wiki page for more information including instructions.

Centralised Logging

I’m currently looking at some centralised logging tools and the following stand out:

  • Octopussy – one I cam across a long time ago but looking at some of the others below it may be past its sell by date?
  • Graylog2 – GSOH (in dating parlance) – “Manage your logs in the dark and have lasers going and make it look like you’re from space.“
  • logstash – “a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs.“
  • Kibana – You have logs. Billions of lines of data. You shipped, dated it, parsed it and stored it. Now what do you do with it? Now you make sense of it… Kibana is an alternative browser based interface for Logstash… that allows you to efficiently search, graph, analyze and otherwise make sense of a mountain of logs.

Kibana has a Bootstrap UI and is written in PHP which immediately bumps it up my list 😉

Monitoring Asterisk via SNMP (with OSS_SNMP)

Over on the company blog, we just announced that we committed a complete Asterisk MIB implementation to OSS_SNMP in addition to a proof of concept Asterisk wallboard to NOCtools which includes active channels, uptime, version information and more.

Full details are on the company post but here’s a snippet of how easy it is to query Asterisk over SNMP:

 $host = new \OSS_SNMP\SNMP( $asteriskIP, $community );
 echo "Asterisk version: " . $host->useAsterisk()->version();
 echo "Active calls: " . $host->useAsterisk()->callsActive();
 echo "Calls processed: " . $host->useAsterisk()->callsProcessed();

And here’s a screenshot of the proof of concept Asterisk wallboard:

[NOCtools Asterisk Wallboard]
NOCtools – Asterisk Wallboard

NOCtools – A Mixed Bag of Tools and Utilities for NOC Engineers

NOCtools is a mixed bag of tools and utilities that are useful for NOC engineers. This project originally started out as a way to highlight and utilise our OSS_SNMP library (a PHP SNMP library for people who hate SNMP).

It since grew into a way to graphically present information on network topology that is normally difficult and cumbersome to do by logging into individual devices. Such information includes a discovered L2 topology by CDP, using this to present rapid-PVST port roles and so forth.

From the company blog:

Today, we are introducing NOCtools which uses this library to provide a number of useful tools including:

  • CDP Neighbours: for a given CDP enabled device, display its CDP neighbours with information and also a graph showing connected ports.
  • CDP L2 Topology: graph the layer 2 network topology based on a recursive crawl of CDP neighbours.
  • RSTP Topology & Port Roles: similar to CDP L2 Topology above but this takes a specific VLAN and identifies and graphs Per-VLAN Spanning Tree port roles.
  • Per-VLAN RSTP Port Roles: a tool that will display the per-VLAN Rapid STP port roles for a given VLAN (or for all VLANs) on a device.
  • Inter-Device VLAN Comparison: a tool that will compare VLANs available (and their respective names) across selected devices allowing you to ensure consistency as well as perform simple security audits.

Follow the links about for screen shots and more details. We are releasing this under a GNU GPL license in the hope that the wider networking community will benefit from them.

A PHP SNMP Library for People Who HATE SNMP, MIBs and OIDs!

I hate SNMP! But I have to use it on a daily basis with my company, Open Solutions.

Don’t get me wrong, it’s an essential tool in the trade of network engineering but it’s also a serious PITA. Finding MIBs, OIBs, making them work, translating them, cross-vendor translations, etc, blah, blah. And then, when you do find what you need, you’ll have forgotten it months later when you need it again.

Anyway, while trying to create some automatic L2 topology graphing tools (via Cisco/Foundry Discovery Protocol for example) and also some per VLAN RSTP tools to show port states, I started writing this library. As I wrote, I realised it was actually very useful and present it here now in the hopes that the wider network engineering community will find it useful and also contribute back MIBs.

An example may best illustrate the library:

First, we need to instantiate an SNMP object with a hostname / IP address and a community string:

$ciscosw = new \OSS\SNMP( $ip, $community );

Assuming the above is a standard Cisco switch, let’s say we want to get an associate array of VLAN names indexed by the VLAN ids:

print_r( $ciscosw->useCisco_VTP()->vlanNames() );

This yields something like the following:

Array (
    [1] => default
    [2] => mgmt
    [100] => cust-widgets
    [1002] => fddi-default
    ...
)

It really is that easy.

See the GitHub project page: https://github.com/opensolutions/OSS_SNMP

What the Hell is INEX? An IXP?

In a few recent posts, I’ve mentioned INEX.

INEX is a neutral, industry-owned association, founded in 1996, that provides IP peering facilities for its members. INEX membership is open to all organisations that can benefit from peering their IP traffic and there are currently 57 members.

INEX can also be considered Ireland’s IP Peering Hub. INEX membership provides high-speed, reliable and resilient IP traffic exchange facilities for both Irish and International organisations, allowing them to route IP traffic efficiently thereby providing faster, more reliable and lower-latency internet access for their customers.

So what the hell is an IXP? Well, Euro-IX commissioned the following, the Internet Revealed: A file about IXPs, a couple of years ago which brilliantly explains IXPs.