I did some reading this morning. Here’s a headline that really caught my attention: “Hospital datacenters: Extinct in 5 years?”
Immediately, I did a double take. I wasn’t surprised at the premise… I was surprised at the timeline.
Here is why I did a double take: Have you ever been in a hospital data center?
I’m not talking about system data centers here. I’m talking about the data centers that we maintain onsite at the hospital. Many of them are in the worst part of the building, and for good reason: they are loud and real estate in a hospital building is hard to come by.
I’ve seen data centers in temporary mobile buildings. You know what I’m talking about -- an all-wood, temporary structure in the desert with a bunch of racks running at high temperature. One carelessly tossed cigarette and the entire hospital’s IT systems are down.
A more common hospital data center is in the basement. They typically have water pipes running across the ceiling and the building sits in some disaster zone.
So, it’s through this lens that I wonder how we can be talking about eliminating hospital data centers in 5 years… the year 2022.
I’d like to offer three reasons that this work shouldn’t be put off until then.
Because it's possible
In 2011, I told the staff at my health system that we were eliminating all data centers in the hospitals. We challenged our vendors to provide a plan to get everything that was needed to run onsite into two racks.
In the end, the largest data center was 18+ racks and the smallest was five. Through hyperconvergence, we were able to get it into a single rack for every hospital.
I’m really not sure what else to say here. This isn’t a move to the cloud, although it could be coupled with a cloud strategy for a more powerful solution. No, this is straight-up consolidation of the environment with the same level of computing, storage, and network capabilities in a tiny package.
A self-contained rack is a lot easier to place than a 500sqft room with specialty power and cooling.
Because of the potential
When hyperconvergence is coupled with a virtualization and cloud strategy, you end up with the budget you never thought you had for disaster recovery.
The budget and capability come from three areas:
- Consolidation forces Virtualization,
- Consolidation reduces maintenance costs, and
- Consolidation concentrates your investment.
Virtualization is the foundation of the cloud. When you can make the compute and storage of your data center portable -- from one machine to another with the click of a button -- you’ve created agility. Now you can begin to experiment with disaster recovery strategies that were not available before.
Virtualization is the building block for disaster recovery.
Maintaining a data center is expensive. Maintaining one rack of equipment? Eh, not so much.
The second area where you find money is in the move to virtualization. There, you end up with automation tools that aren’t available in the physical server world. Automation is the only way to keep your servers patched and secure.
Lower cost and a more secure environment.
Finally, by concentrating your investment, you free up additional budget money for higher utility items. You can buy a CRAC (computer room air conditioner), or hire a data center cleaning specialist to keep the space below the raised floor clean. Or you can invest in training and resources that are more closely tied to the mission of the organization.
Once complete, you can do things that weren’t possible before at a fraction of the cost. As an example, let’s imagine that you moved the entire hospital down the street for some reason (say there was a disaster). In this case, you could move one single rack and the entire compute environment could follow.
Try that with a fixed-asset, 16-rack data center.
Because of the consequences
I’ve heard three stories this week of hospitals being without IT functionality for more than 24 hours, due to outages. No, we aren’t talking about data centers in Florida and Texas -- these are data centers in places not affected by natural disaster.
I’ve heard of virtual desktop environments that are so poorly put together, clinicians aren’t sure if their workstation will work from day-to-day. Can we possibly be this busy with EHR and Population Health projects that we have forgotten to do the basic blocking and tackling of the IT organization?
When you consolidate, you are forced to clean things up. For example, we found that we had hundreds of server images, which we reduced to about six. When you do these cleanup projects, you are forced to update the skill sets of your staff, identify the lack of standards, and address the holes in your operation.
In short, you are required to question what you’ve been doing all along. It’s hard work. It’s the work that only the IT department can do for the organization.
Why I believe these projects aren't being done
There are three primary reasons that I think these projects aren’t commonplace already. They are:
- Skill Set
The budget for this overhaul should be sourced from your maintenance budget. When I hear of a health system having over 50% of the equipment in their data centers past end of life, I consider that malpractice. The job of the CIO is to ensure that this doesn’t happen.
You either need to do the hard work to get the budget, or come up with a different strategy to provide compute and storage to the organization. There is no other option.
The complexity of this project is mostly due to poor IT practices within health systems. When I asked for an application catalog at my last organization, it took over 3 months to put it together. Then, we found 70 more applications strewn across the enterprise over the next year. The CMDB was non-existent and we managed servers based on the skill of the person, not based on standard practices.
Do this for any period of time and you have no idea what you actually have.
If your organization isn’t intentional about training, your staff is probably not ready for the next generation of architecture that is coming down the pike. The biggest change is that siloed support isn’t all that helpful anymore; you need people that know the whole stack.
It used to be that you had a network person, a storage person, and a compute person. In a virtualized world, your people truly need to have a working understanding of the entire stack
What you can do
There are ways that you can proactively mitigate these issues, before they become a thorn in your side.
Educate your team on hyperconvergence and cloud. Evaluate how you are spending your data center refresh budget. Look at those budget items that would go away if you were able to fit into one rack in the hospital.
Consider the value of automation on things like server patching, provisioning, and disaster recovery. Ask your team for a list of all your applications or, even worse, all the data integrations with third parties.
Turn over the rocks and see if you feel comfortable about the stability of your environment. Then ask yourself, ”Can we wait five whole years to provide this basic service to the hospital?”