Blinky lights

Like the vast majority of google engineers, I'd never seen one of our data centers in person. For one thing, they're mostly not easy to get to - they're big facilities which need a lot of space and generally aren't close to things - and for another, they don't really do tours. We had an appointment in the Council Bluffs facility for 3pm today, and four days of driving later, here we are.

Chris Russell was kind enough to show us around the facility there, and [ALL INTERESTING DETAILS REDACTED]. It was genuinely stunning, like seeing the future in person.

A datacenter tech helped us locate one of the GWS, which was sitting blinking on its rack like all of its many fellows, happily processing [REDACTED] queries per second. Until today, I have to admit, whenever I imagined a gws I always imagined it as a little white square on a diagram with "GWS" written on it, underneath boxes for frontends and network gear, in front of boxes for the rest of websearch. Now they all have faces, it seems so much more personal. Even if it's mostly all the same face.

As an SRE I think about gws in terms of number of units and machines, geographical facilities, machine configurations, performance profile, and so on, and I spend a lot of my time worrying about their individual health, and working on ways to make them all stronger and tougher. But mostly I think about multitudes. The rest of the group, all engineers, I suspect are more used to thinking about gws as a singular entity, the one they code and build and run on their desktops. Seeing the facility is different. As well as the machines themselves, it's a reminder of all the construction and physical labor and day to day work of the techs which makes it possible for that code and all those jobs to come to life.

I think everyone resisted the overwhelming urge to poke the machines or maybe unplug something. But we were all thinking it.