Well, we moved our two companies this past weekend to a new ISP.  One of them went relatively smoothly.  Another did not.  In fact, I'm still in there, blogging at you from betwixt rows of sleek black 8 ft high cabinets and the ever present whir and hum of countless servers and other devices.  Unfortunately the plethora of troubleshooting that befell me was largely intermittent.  The entire cabinet would lose its connection to the outside world at times, and at others the connectivity would remain but outbound DNS lookups would fail and some inbound things would work and some wouldn't.  So both the laptop I've had plugged in as well as some of our servers that connect outbound by name were negatively affected.  When you do trial and error in this situation, when things start working again, is it because of the change you made, or because, as is the nature of intermittency, that just happens to have been when things naturally started working again on their own?  So we made changes.  We laughed.  We cried.  We got dunkin donuts and coffee.  We swapped cables and stories.  It was the crappy of times and even crappier of times.  We tested the lines, we rebooted, we tried different ports.  At one point I even plugged my laptop directly into the main network drop for the cabinet.  That, of course, had no problem, so we gradually narrowed it down to the pix firewalls we have.  Thankfully we have two, a primary and a backup secondary.  With all options exhausted, I downed the primary and let the secondary takeover.  From then on everything was groovy.  So after an hour of that or so, I swapped cables on the primary and brought that up (so if there's still problems, its not the cables).  55 minutes so far.  I have to give at least 1.5 hours because that's how long it was up before I had problems yesterday.  If it's all good after 2 hours, then all the changes we made today are probably good.  At the very least I'll be able to sleep knowing I put in my all, and I pushed back my flight home a day so I can be around tomorrow.  Hopefully it will be an uneventful night and tomorrow I will just lie around the hotel and monitor.  If I have to come back tomorrow I just might scream, and in the data center, nobody can hear you scream, or hear you fart for that matter.  Or even smell you fart since the air is constantly being circulated.  Big Beef and Bean Burrito, here I come.