Like the vast majority of google engineers, I'd never seen one of our data centers in person. For one thing, they're mostly not easy to get to - they're big facilities which need a lot of space and generally aren't close to things - and for another, they don't really do tours. We had an appointment in the Council Bluffs facility for 3pm today, and four days of driving later, here we are.
Chris Russell was kind enough to show us around the facility there, and [ALL INTERESTING DETAILS REDACTED]. It was genuinely stunning, like seeing the future in person.
A datacenter tech helped us locate one of the GWS, which was sitting blinking on its rack like all of its many fellows, happily processing [REDACTED] queries per second. Until today, I have to admit, whenever I imagined a gws I always imagined it as a little white square on a diagram with "GWS" written on it, underneath boxes for frontends and network gear, in front of boxes for the rest of websearch. Now they all have faces, it seems so much more personal. Even if it's mostly all the same face.
As an SRE I think about gws in terms of number of units and machines, geographical facilities, machine configurations, performance profile, and so on, and I spend a lot of my time worrying about their individual health, and working on ways to make them all stronger and tougher. But mostly I think about multitudes. The rest of the group, all engineers, I suspect are more used to thinking about gws as a singular entity, the one they code and build and run on their desktops. Seeing the facility is different. As well as the machines themselves, it's a reminder of all the construction and physical labor and day to day work of the techs which makes it possible for that code and all those jobs to come to life.
I think everyone resisted the overwhelming urge to poke the machines or maybe unplug something. But we were all thinking it.
GWS team is taking a road trip. Sixteen unruly miscreants led by Capt. Ahab will descend upon Pittsburgh in two RVs. We will brave the perils of the high seas from Mountain View, CA. Many more derelicts will join us for much revelry. Join us for the adventure.
What could possibly go wrong?
Saturday, October 16, 2010
Arthur Bryant's
Friday, October 15, 2010
Kansas!
Fourth state of the day! Meat grows all around me. This cursèd game with symbols has devolved into a drinking game. We are eating in an airplane hangar tonight that has been converted into a steakhouse. Insanity has gripped this death R.V. Lord help us all
Technical Difficulties
RV #1 (callsign Primo) endured various technological hardships today. We let our gas tank get too low, putting the generator into shock and leaving us without air conditioning through the midday sun in Oklahoma. We eventually debugged the RV, cooled off and were able to charge our electronics.The internets also failed us. At first we thought Jinnah's "unlimited" sprint mobile data plan had been maxed out by too much starcraft the past few days. I spoke with a tech stop phone support guy (discovered that cell connection from Oklahoma to India is rather weak and that Jinnah's ldap is pronounded "Juliet Delta Hotel") and escalated the issue by explaining that we had *SREs* without internet. It turns out, our data plan was fine, and the same weak phone connection was responsible for keeping us internetless.
We've prevailed despite the trials. Justin, Dave, and Vince have even succeeded in blending in with the locals.


Up next: Steak for dinner in Wichita and BBQ for breakfast in Kansas City. Doug, Neal and I are catatonic with excitement.
Oklahoma City and the Game With Strange Symbols

We're now passing through Oklahoma City during rush hour ... with a man at the wheel who has not yet driven the R.V. The occasional car is honking at us. I fear for my life, and so begin preparations for my imminent demise.
Others prepare for their fate in a different way. They play a strange game with symbols. New rules are being continually created. The net result is that the collective score continues to decline. With all hope of victory extinguished, we have reverted back to drinking scotch.
If anyone knows a place to eat in Wichita, let us know ... in the event that neither the R.V. nor this game consume our souls.
Subscribe to:
Posts (Atom)


