Fully automating server infrastructure for most is the holy grail for a majority of sysadmins (or whatever the modern day job title is). Not having to click through a wizard, manually apply IP addresses or patch a machine, is attractive for anyone who has sat around as Microsoft’s latest patch update trickles through your virtual NIC. But are you doing it for the right reasons, and are you making sure you are getting the most value out of your automation?
I’ve spent plenty of time in the past year reading Twitter and LinkedIn posts, blogs and technical articles on the best way of achieving end-to-end provisioning and configuration of servers. And shock horror, each one was completely different, each of the authors wanted to achieve different things but none of them we’re complete in my eyes. Having said that, I don’t think I’ve ever written a complete blog post either, so no pointing fingers from me!
I think the big reason the blogs never manage to meet the holy grail of automation for me, is because I’m boring, and I feel lot of them miss out critical steps, typically around governance or asset management. I know those two terms are possibly the MOST boring terms in the history of IT, but honestly – they are king. I’m fortunate to work with a wonderful Head of Governance and Risk @ Stagecoach, and she’d be the first to admit her job is far from sexy, but what she brings to the table is irreplacable.
So, what benefits should provisioning and configuration of servers bring to your organisation?
Are your teams getting bombared from all areas of your business asking them to deploy servers, or containers for the latest service?
Are your dev’s asking for environments to be rebuilt in a whim?
Do you want to abstract your server team from the mundane tasks that is building a server?
Do your like-minded server suffer badly from configuration drift, and your users are getting different experiences dependant on which web server they hit?
All the above questions are fantastic reasons to go and begin your journey, but rather than do what most techs do (which is get all excited when you do something cool in Terraform) and not think about a workflow with tangible outputs. Stop, take a step back, and consider what you want to achieve, and that means considering the impact on other teams.
To get you started, I’ve put together a little checklist as to what you should consider when you’re looking at server provisioning and config management. These are not an exhausive list, and nor are some of the steps mandatory, but I would imagine most readers of this blog can relate to some, if not all of these steps. Hopefully it gives just one person food for thought when they start on their automation masterplan!
- Workflow Trigger
This could be a self-service ticket from your ITSM toolset (Servicenow, etc) or it could be a response to high demand from your monitoring tool – either way, the idea is that this trigger is not manual
2. Hardware Configuration
Whether its vSphere, AWS or Azure, the hardware configuration is one of those manual steps. Pretty much everyone is used to configuring the number of CPUs/cores, memory, storage and network cards. Nothing unusual here
3. Operating System
So typically this would be an install of a commonly used OS, such as Ubuntu, Windows, Redhat, etc. In the AWS world, this would be the choose of an AMI, in the vSphere world this would be the cloning of a template, in Azure this could be a Marketplace image.
4. Domain Join & Server Name
This step is probably only relevant to those deploying Windows servers, but the joining of a machine to a domain for the purposes of authentication, etc is a mandatory step for those reliant of Active Directory. Server naming I always find useful, especially if its matches a particular pattern that I use for querying logs, or running reports later on.
5. Operating System Patches
Like most businesses, you have to adhere to your compliance policies, whether that be a world-wide standard, or an agreement internally with your stakeholders. Make sure those patches are installed and all post-patch service tests are completed.
6. Mandatory Software
Most mandatory software will be provided by the fun sponge department, also known as Information Security. I jest of course, I love Infosec… honest! But this would typically be software such as your AV client, any vulnerability scanning, monitoring or backup agents, etc.
7. Application & Services
So this step is the real reason you built this whole automation workflow. This is the main software, or the service deployment which the server exists for. This could be a database, web tier, or whatever. Its vital that this deployment is version controlled and easily upgraded, so making sure this is deployed inline with your CD (Continuous Delivery) framework is critical.
8. System State
So at this stage, you’ve got a working service running on your new server, everything is working a-ok. Now you need to make sure it stays that way, and in line with any other like-minded servers within your stack. This could be 1 of many web boxes behind a load balancer or an AD domain controller, they all should have the same state.
Do you have a one-off script that runs to ensure this machine is added into your monitoring platform? And I don’t just mean hardware monitoring, I mean the full stack – critical services, thresholds, folder monitoring, etc.
Are you shipping your logs to your SIEM? Have you ran a vulnerability scan and ensured that any high or critical CVSS vulnerabilities have been remediated? Do you install application controls apps to prevent unauthorised apps from running? These things need doing
11. Asset Management
Yep, I know – BOOOOOORING! But its important for those pesky service folk that they know this server exists, so they can report on it, log incident, problem and change records against it, etc. And your procurement team will want to know about it too, especially if requires spend for you to run it, or licenses are consumed by it. This is a hugely critical step which is very frequently overlooked.
As I’ve said above, there are far more elements to this, documentation of new services, transition documents, approvals to bring to live, the list goes on. But on a really basic level, if you’re wanting to get yourself started in the world of provisioning and configuration management, this is a good starter for ten.