The Server Farm

I was a Database Administrator (DBA) in my previous job. We were a highly specialized shop in most respects, which means that different individuals handled specific tasks. For example, as a DBA I was not allowed to set up virtual servers or allocate resources to existing servers, and I had limited authority to make configuration changes at the operating system level. As a result, I sometimes found it difficult to guarantee system integrity. This lead to a lot of personal stress, so as a stress-relief exercise I made light of a particular series of events that occurred during my last year of employment. I shared this story with my coworkers in bits and pieces as the events unfolded and they found it entertaining, so I thought I'd share it with the world at large.

I've included some additional commentary to help explain the circumstances a little better.

Mainframe had a database that was one of the primary sources of information used by my employer.  Directly accessing the mainframe for daily reports was impractical because of performance issues.  I used the DRDA gateway to copy the relevant data into the Prod database on a regular basis and ran the reports from there.

The conversation between me and the Network Person went something like this:

NP:  "Are you using Test?"

Me:  "No.  I am not doing any testing today.  Do you need to take it off-line?"

NP:  "Yes."

I interpreted this conversation to mean "We, the Network People, need to reboot Test because we are using it for some nefarious purpose of our own, such as World Domination."  It was a perfectly reasonable assumption since both the Network People and I used this particular server for a variety of testing purposes.  What the Network Person really meant was, "We, the Network People, are going to reformat Test's hard drive and turn this machine into a menial Zombie Slave."

Reports was brought on-line for a single user who did a lot of complex queries and reporting.  It was determined by our administration that this individual should have a separate database in which to work.  Reports was supposed to be simple:  one machine would have all of the necessary tools, including the database, running in an easy-to-use GUI operating system.

I never did find out why Reports and Mainframe couldn't get along.

Most of Dev was set up to parallel Prod, so it was a fairly easy operation to "graft" Dev to Prod. Still, it made me look like a Genius Miracle Worker to most of my peers. (Those who didn't think I was a Genius Miracle Worker probably didn't understand the setup, anyway.)

Starting at this point, Reports was only used for hosting the database. The reporting tools were installed on a different server.

There was a long period of time between the slide above and the slide below. We had to coast along like this for a while because other projects had to take precedence.

Certain batch processes on the mainframe would lock entire tables, causing Prod's jobs that read those tables to time out. The scheduling of the mainframe batch processes seemed to change unpredictably.

Well, that's the end as far as I'm concerned. My successor will need to pick up the standard and carry it forward into the battle now. I wish him or her all the best.