Building Out a Security Skills Matrix

Thursday, February 24, 2011

Brad Bemis

B44a73900ca3197c2d8f148e303b3faa

It was still dark this morning when I opened my e-mail. The first thing that caught my eye was a request on the CISSP Forum for information on the skills needed to operate an IPS.

I posted a quick response to the request, and then flipped the guy a copy of an old skills matrix I had developed as the manager of a security team a while back. Of course, this made me think ‘hmmm, I wonder if anyone else might benefit from this?’.

Obviously I’d be a sucker to post something as cool as a complete skills matrix to the Internet for absolutely free, so instead I’ll talk about the approach I used and give you a quick peek at what the matrix looks like (headers only I’m afraid). Maybe you’ll find it useful, maybe not.

However, as the manager of a security operations team, I needed a way to track the current skills/knowledge levels of my team, and its progress over time. That’s what this matrix did for me – and it could do the same for you to – IF you’re willing to take the time to build one that is.

To make this a little easier on me (since it’s still dark outside, and I haven’t quite shaken the sleep from my brain yet), I’m going to include a portion of the team training plan I wrote to accompany the skills matrix (you’re not going to get that for free either I’m afraid ;-).

Here is the relevant chunk from the training plan – I’ll explain more after the break:

In xxx of xxx a detailed skills assessment was created for the xxx team. This skills assessment covers 7 key areas of knowledge and skill (defined as skill families) required to successfully fulfill our team charter.

The following skill families include a series of specific ‘knowledge measures’ used to gauge current skill levels at a granular level:

• Soft Skills (17 knowledge measures)
• General Networking (60 knowledge measures)
• General Security (111 knowledge measures)
• Enterprise Firewalls (49 knowledge measures)
• Enterprise VPNs (35 knowledge measures)
• Enterprise IDS/IPS (41 knowledge measures)
• Enterprise Web Filtering (22 knowledge measures)

[Unnecessary section removed, but it covered other technologies I hadn’t gotten around to mapping yet]

Each xxx team member was tasked with completing the xxx Skills Assessment template by rating themselves on a scale of 1 through 5 against each knowledge measure. The available responses to the knowledge measure questions were:

• 1 – No
• 2 – With difficulty
• 3 – Mostly
• 4 – Yes
• 5 - Absolutely

If you’re curious to know what this actually looks like, here’s your sneak peek – don’t blink, you might miss it:

image

Yeah, this sample section covers ‘soft skills’ – sorry about that, but it was the first item on the list. If you're team is more technology-centric though, I might add that soft skills are often overlooked as an essential component for creating well-rounded security professionals in the workplace (especially on operational teams). I’ll be writing a future blog post on soft skills for security professionals a bit later on down the road though, so no need to fret.

So let’s talk about the breakdown of elements here:

Skill Families: This is the term I use to describe a collection of specific knowledge areas or required skills that are related in some way or another. For me, this is usually based on the categories or ‘buckets’ of work that my team is responsible for (this isn’t the first time I’ve used this kind of training matrix for a team I managed).

From the training plan snipet above you’ll see that I had included skill families for Soft Skills, General Networking, and General Security – these just covered the basics of being a security professional from my perspective.

Next were very specific technologies that were present in the environment for which we had operational responsibilities. It’s an incomplete list in the sample (because I just hadn’t gotten around to mapping out some of the more minor technologies we owned), but you get the idea.

Knowledge Measures: In the context of this skills matrix, I wanted a way for a person to rate their current level of skill or knowledge against a specific measure. I did this by asking a very basic question for each line item.

For instance, ‘Are you good at communicating with others?’ is the knowledge measure for Interpersonal Communications. This only needed a little explanation to the team – “don’t over think the question – just say 1 (no), 2 (with difficulty), 3 (mostly), 4 (yes) or 5 (absolutely). Pretty easy right?” The knowledge measure question needs to be kept really really simply – don’t try to make it more complicated than it needs to be.

Knowledge Rating: We already covered this, but I wanted to capure how much the person felt they ‘knew’ about the topic – what their comfort levels were as far as talking about the topic, or explaining it to someone. The scale was 1 to 5. Any more is too confusing, but you could cut this down to just 3 options to simplify it if you need to.

Experience Rating: Knowledge is one thing, but experience is another entirely. As a security professional you may have read up on firewalls, thoroughly absorbing anything and everything on the subject, but if you don’t have the experience sitting in front of a firewall console – you’ve only got half of what you need to avoid bringing down the network.

Experience was also measured on a 1 to 5 scale with 1 (none – i.e. NO experience at all), 2 (Less than 1 year), 3 (1 to 3 years), 4 (3 to 5 years), 5 (5 years or more). I think these are pretty good experience ranges, but you should see what works best for your needs. Again, keep it simple and clear – don’t make things any more complicated than they need to be.

Interest Level: You may look at this one and say ‘why is that needed?’. It’s not really, but this is one of those little thing that serves as a good example of the difference between ‘management’ and ‘leadership’. I can certainly manage my training program if I know where your skills and experience levels are on a given topic – but wanting to know how interested you are in that topic is going to give me some better insight into you as a person. It’s also going to help me see how to best use your passions, or identify places where I may need to motivate or support you a bit more.

Don’t underestimate the power of someones level of interest as a training tool, there’s a lot to be gained by asking for this information. The rating system for interest levels is 1 to 5 as well with; 1 (None – not at all interested), 2 (a little interested), 3 (some interest - yes, it’s interesting), 4 (Very interested), and 5 (ahem – well, this is just me being me really. OMGTISFC stands for ‘Oh My God, This Is So Fracking Cool!’ – what can I say, I’m a BSG fan).

Individual Skills: I kind of skipped over it, but under each skill family you have the specific individual skills that you are writing knowledge measures for.  Where do they come from? Ummm, I kinda made them up. Here’s what I suggest to you though – first of all, look at your teams charter (if you don’t have one, write one). Figure out what your team has clear direct ownership of, and what your team has indirect influence over.

For each area that you identify, start breaking down the individual components of what people need to know and/or be able to do in relation to that area. If it’s a technology family, you can look through the manuals that come with the hardware, software, tools, etc. to figure out which things apply in your environment, and then jot them down. This is really the biggest and most important part of the entire process. 

You need to identify the right skills to make sure that you are measuring the right things. Be flexible and open to changing things around as you discover what your team really does or doesn’t need to know or do – and again, keep it simple. Go to a level of granularity that feels right for each item – probably at the task level. It’s up to you though.

Once you have everything plugged into your nifty little spreadsheet, you can add a few formulas here and there to do some calculations. For instance, I calculate the average of all skills for each skill family, then average out the results across all skill families. This is important because it tells me what I need to offer the team as part of my training plan.

If I look across my entire team and see an average rating of a 2.2 in firewalls, and my team owns firewalls, I may want to start doing some training on them. You can use the numbers to help establish priorities to – this is often needed when you have a finite training budget and need to be smart about where you spent your money.

The last thing we’re going to talk about is the interval period between assessments. This is a tool to understand where your team is at today and what you need to focus your teams training on. But it’s also a tool that can be used to monitor and demonstrate progress.

Have your people update the matrix on a regularly established interval – tie it to a quarterly or six month review cycle if you can. I always required updates as part of a quarterly cycle - my training plans and budget came in quarterly chunks. Personally I think annually would be too long of a time period in between assessment; it really limits your ability to make good use of the tool.  Once every six months might work, but I still recommend quarterly.

If you want to go the other way with this, you could ask for monthly updates – the problem is, your worksheet is probably going to get quite large, and the time required to update it may be significant (half an hour to an hour or so per person). Monthly may just be too much of a time and resource burden. Again, quarterly always felt pretty good to me.

There’s a lot more to say on the subject of training and training plans, but hopefully this gives you some insight into how to build out a good skills matrix, and explains why you might want to take the time to do so. In future posts we’ll cover more on this topic and explore other tools that can help you build and train your security team.

‘till then, enjoy…

Cross posted from http://www.secureitexpert.com

Possibly Related Articles:
5407
Policy
Management Employees Information Security Professional Security Skills Matrix Skill Set
Post Rating I Like this!
12b78bed0e9cd7b675706d1b8866cc00
Brian Ford Excellent essay Brad! Very clear and well thought out.
1298740690
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.