Security awareness encompasses a range of factors that oft times are not easily measurable, unlike other technical aspects of securing the digital perimeter of a business. It involves psychological and social aspects of the employees which in turn drives behavior. How do security professionals go about measuring the success of their awareness campaigns that deal largely with human nature?
Join us for our upcoming Community live with Security and Crime Psychologist Nadine Michaelides of Anima People who brings her insights from her own work in providing evaluations on the effectiveness of security awareness campaigns.
What does Evaluating an Awareness Program Look Like?
While an in-depth evaluation covers many aspects of a program, Nadine provided us some key areas to consider. The first is doing a human factor's audit that includes looking at the current tools in use, how the workforce is being engaged, and identifying motivations, barriers to change, and reasons employees may not be security conscious or adopting behaviors the organization wants to be adopted.
From the insights gained in the audit a strategy can be built, but she noted that this step is often where organizations struggle. This is in part due to a misperception often held by many companies viewing an awareness program as simply finding the right computer-based training platform when in reality a successful program should aim to match the complexity of the humans who make up their workforce. To achieve this, requires a multifaceted approach involving a variety of research into the organization.
Awareness is not the point. The point is behavior change... It's really important to recognize that awareness does not mean behavior change.
What are Some Common Barriers?
Mismatched solution for the culture - while there are many great solutions available for computer-based awareness training, it's important to consider how culturally relevant the content is. Ensuring the training content is engaging is important but it's not the only factor to consider when choosing a solution. If the context of the information doesn't relate culturally this can be a factor for employees "checking out" making the training ineffective.
As Nadine has seen, "There are little cultural factors...Such as the backgrounds in the training don't really match their own background, so they're disconnected from the training. It doesn't mean anything to them. So, therefore, it becomes a tick-box exercise. They lose their audience from the first moment often...It's important that the training is considered quite carefully as to whether it matches their audience."
Technical barriers to the business - To quote Naomi Buckwalter in a recent post on LinkedIn, "Security is a service to the business. If the business cannot function because security is too strict, then there is too much security." The ripple effect causes employees to look for workarounds of security barriers to fulfill their own responsibilities. Until these technical barriers are addressed in a way that supports work as opposed to preventing it, no amount of training will prove effective.
Disengaged workforce - When there is a lack of loyalty, commitment, and trust, there's moral disengagement which is a definite factor that affects an awareness program's effectiveness. For those employees who are not particularly motivated to fulfill any tasks from the employer that fall outside of their job description, security can fall into this category.
A disengaged workforce is a big challenge to address but identifying it will help provide where other approaches or creative solutions may need to be considered for a more holistic response.
It's also important to look for any potential 'undermining' of the security program - while it may be unintentional it can undo any behavior change your working to effect. Check out our conversation with Amy Dearwester on the topic of Undermining Awareness.
There's a big assumption that if we give people enough information in a variety of ways, then they will want to do it. And the reality is quite different.
KPIs
For the in-depth evaluations that Nadine and her team provide to organizations the list of KPIs they look at was too long for our short discussion. However, she recommended security awareness managers consider both qualitative and quantitative data to get the most accurate view when evaluating a security awareness program.
It's important to consider the diversity of an organization's workforce in terms of learning and communication as well. "When you're doing a strategy, bear in mind you've got neurodivergent people, you've got people that perhaps don't have very high literacy, you've got people that engage better through video, other people through text, some people need to download as well as upload - you have so many different types of people. So respecting that is really important.
But understanding that requires investigation. You can't just guess that, you need to know that."
How does one go about knowing the varied types of people and their styles of learning and communicating? Through surveys and interviews that include a focus on what was highlighted earlier - looking for any barriers to motivation, psychological views or technical processes that are lacking. Having these KPIs as part of the research will help you apply a strategy that is much more effective and goes beyond a tick-box exercise to support your company's overall security posture.
As Nadine emphasized, "it's not just awareness, it's supporting individuals throughout that journey [of behavior change]. And it can be as simple as including a reporting mechanism, or it can be as simple as training the manager to understand how to ask the right questions or identify if there's been issues within their teams."
Tone of Voice Is Important
When evaluating a program it's also important to consider the tone of voice that the overall messaging has. There is much research that demonstrates most fear-inducing messaging is counterproductive and can even lead to a sort of paralysis when it comes to reporting or other security behaviors.
Reframe Insider Threat
When speaking about insider threat some companies squirm a bit as they've worked hard on building a company culture of trust and support for their employees. Speaking about insider threats, for some, can be perceived as contrary to the culture and atmosphere they've worked to cultivate. However, it doesn't have to be that way.
Nadine feels a better approach to addressing Insider Risk is by adjusting how we understand it. She recommends identifying the future security champions as well as any detractors to security as she says it's a similar kind of assessment. In recognizing who the promoters of security are these individuals will ultimately complement any insider risk management program.
In fact, she suggests incorporating this level of assessment and identification into the hiring process, "I think it would be quite useful to understand right from the recruitment and selection stage, who are the people that are really going to help us here in bringing that message up to employees about the importance of security culture."
What about Measuring Security Culture?
Integrating the security assessment from the very beginning of an employee's experience with the company definitely helps contribute to an overall security culture that is embedded into the corporate culture. However, Nadine cautions a little on leaning hard into the 'trending topic' of security culture without proper evaluation of what is meant by it for your company.
"I'm a little bit critical of the word culture. I use it myself because it's a great word. But what's important...when we try to measure culture, is to be sure you understand what you mean by culture, because the definition of culture changes from different schools of thought, and different organizations see it very differently.
So the first thing before you try and measure your security culture, you really need to pin down, what does security culture mean?
Otherwise, those KPIs are just meaningless. That's why I prefer to use risk, because everyone understands what risk is, and it's much easier to define and measure, essentially."
How To Communicate Our Findings To the Decision Makers?
Once you've evaluated your security awareness program, identified areas for improvement, and developed a strategy to enhance it, there remains the challenge of presenting to the decision makers. While recommendations may vary depending on the level of investigation and type of data gathered, Nadine recommends:
- Communicate findings in as visual a manner as possible - dashboards, graphs, visualizing statistics, etc.
- Use the KPIs to guide the key points of the presentation
- Focus on just four or five main points
- Present those same key points for each meeting - this will help shareholders know what to expect and better understand the progress (or lack thereof) to prompt action
Nadine's Parting Advice:
Challenge the lingo - awareness, culture, behavior. What does it mean? How can we measure it? Do we all agree that that's what it means? Do we all agree that we're measuring awareness? Do we just want to inform people with a load of information or do we want to involve them and inspire them to become security champions?
Connect with Nadine Michaelides on LinkedIn and while you're there check out our Security Awareness Manager community.
Nadine's Resources from Anima People:
The Psychology of Onboarding: Optimising the new-starter experience in the digital age
Cybersecurity awareness campaigns that drive behaviour change (2022)
More Resources from Wizer
Customizing Security Awareness Campaigns with Dr. Alexander Kharlamov
Building a Winning Security Awareness Program - Interview with Dennis Legori
Creating an Ambassador Program for Security Awareness Programs - Interview with Nandita Bery
Building Healthy Security Cultures - Interview with Nadja el Fertasi
Security Awareness Community Manager's Hub
