Time To Settle The Score: Why Your TPRM Strategy Needs More Than Questionnaires And Tick-Boxes

It’s hardly a novel observation to suggest that we live in confusing and contrary times. Lots of things about our present shared environments make very little sense, so singling any one tiny example out might seem pointless. But there is something emblematic of our socio-political-cultural moment in the strange way we all seem to react to questionnaires.

As consumers we appear to love them. As attention spans atrophy and an audience for traditional written “content” becomes ever harder for publishers to find, no online resource is going to struggle for clicks and eyeballs when it posts some kind of “what Pokémon character/domesticated animal/type of cheese/superhero’s sidekick are you?” quizlet. Everyone wants to have a go, regardless of whether there’s any kind of prize involved, and completely oblivious to small print that may say, by participating, we’re giving permission for 174 tracking services to monitor our web browsing activity for the next 100 years. But when it’s our job to fill in questionnaires sent by our suppliers or our customers, and quizzing us on our risk appetite, our security processes and policies, and on how safe and reliable a partner we may be in an ongoing and mutually beneficial business relationship, questionnaires seem to become the last thing anybody wants to have anything to do with.

And yet, according to many views expressed by a high-level group of CISOs, BISOs and other senior cybersecurity leaders called together by risk-management specialists Diligent for a RANT roundtable in Manchester, questionnaires remain the first and best option for many businesses when it comes to assessing and assuring the security of their usually extensive supply chains. But why is that? Surely there’s got to be something out there that works better, is more effective, more responsive, less arduous and should be more reliable a gauge of a partner’s security policies and practices than the much-decried and almost entirely unloved extended tick-box exercises that most businesses insist on their suppliers submitting to?

“My perspective on all this has grown over the years,” said RANT’s guest-host for the evening, Colin Farrell, currently head of security audit at the Office for National Statistics but who, in previous roles, has worked extensively in regulatory audit capacities and spent time on staff at the Information Commissioner’s Office. As such, his overview of supply-chain risk is extensive and his perspective broad. The trends he’s noticed are therefore noteworthy.

   “The interesting thing about third-party risk management is that it’s the only aspect of security I can think of where everybody’s basically doing the same thing,” he said. “It’s different shades of grey – but everyone uses supplier questionnaires. They look a bit different, the processes are different, and I’m sure there are examples of industries this doesn’t apply to. But questionnaires exist for a reason – they work. So what interests me tonight is, what are those different methods? And has anyone found anything that works better?”

Ready Or Not

The first theme to emerge from the discussion was that questionnaires, like any tool, work best when deployed with precision, care and planning, rather than reflexively reached for as a blunt solve-it-all-somehow instrument. And while some attendees flagged up exceptions, and some suggested a few ways of obtaining the necessary assurance that may not use questionnaires as the sole or central means of obtaining information, for most security and audit managers, the best way of making questionnaires work well lies in ensuring they’re sent out – and/or responded to – only after first asking and answering some simple, basic and internal questions.

   “When you onboard a new supplier you need to understand the service they’re supplying – and you need to have an exit plan, because without one you can’t even onboard them,” one security leader argued. “To me, that’s proper risk management. It shouldn’t be, ‘Have you got this policy and that policy? Yeah? Then we’ll onboard you.’ No. It’s not about a tick in a box.”

“Not all suppliers are equal, are they?” another leader mused, pointing out that a firm that supplies pens to an office will not need to demonstrate the same kind of security assurance as a financial partner or a company whose systems are required to connect to the corporate network. “What we try to do,” they added, “is not ask everyone the same questions. There are screening questions, then what we ask after those is different.”

Very quickly, the conversation turned to not just understanding the nature of the relationship between the companies and appreciating the differences between different kinds of suppliers, but to means and methods of accurately and sensibly assessing risk in the round.

   “What are the things on top that you need to do to understand what the true risk is?” one expert asked, partly rhetorically, partly genuinely and literally. “You need an external view of how controls are working, and you need that on a continuous basis. Then you’ve got to understand different suppliers depending on the risk they represent to you based on who they are and what they do for you. You should,” they added, “ask the same questions to all of them. [But] what do you want to get out of it? The questionnaire gives you something, but you need an assessment of what they’re actually doing.”

The Mask

At the end of the discussion, Diligent’s governance, risk and compliance sales director, Tom Ryan, humble-bragged that neither he nor his colleague, director of sales Chlōe Dellow, had mentioned AI once. But it wouldn’t be a cybersecurity conversation in 2026 without someone raising the spectre of autonomous agents, large-language models and generative so-called “artificial intelligence” – and, inevitably, this contentiously inescapable technology was evoked quite early in the evening. But for once, the impact it seems to be having – at least for some businesses – has been positive in terms of how it has redirected discussions within companies during examinations of supply-chain risk.

   “AI has helped in this respect with us,” one attendee said, noting that their product is software, most of their suppliers supply software, and AI is contained within most of those supplied software products. “It has forced governance across the whole company. Now we’ve got people who are looking at the impact across the whole enterprise. Legal are doing a deeper dive on terms and conditions, and security is now a part of that process. We’re not driving it anymore – which is nice. There’s no escaping it, so it’s going through a more rigorous compliance process.”

But in other sectors – perhaps particularly those where the product or service being sold is not software – such deepened corporate understanding may be slower to coalesce. And, in any case, it will still be necessary to ask the right questions if anyone hopes to obtain answers that are accurate, insightful and reliable. Understanding the nature of the relationship remains fundamental and unavoidable.

   “You’ve got to look at the non-standard supplier,” one security leader, who works in a regulated industry, said. “We have to break things down to: What companies am I mandated to use? What are the regulated companies I have to work with? And who is there outside of that who I still need to be on top of? It’s all about piecing that all together to get a fuller view. What risks are you willing to take? That’s not just cybersecurity risks – it’s all the other risks too.”

Freestyle Interlude

One of those other risks – unavoidably, if somewhat metatextually – lies in whether or not you can trust the responses you get from your questionnaires.

   “Sometimes,” one CISO suggested, people give certain answers because “they want to keep the business on side. Whereas in your head you’re thinking, ‘This is complete BS, and if the regulator were to come along and look at this, we’d be screwed.’ The biggest problem we see is people being able to explain to the business what the risk is, in language they understand. And what do you do next with all that information? You’ve got it, but often, it just drops down a hole.”

And of course, the questionnaire experience usually works in two directions, not just one: most firms, as well as attempting to ensure adequate and acceptable security policies are in place with suppliers, will be on the receiving end of similar approaches, responding to similar concerns, from the companies they supply their products and services to. Making sure the business can do that is just as important, and the theme of responses being coloured by the likely perceptions of the intended audience is resonant here too.

   “I want to get away from the idea that [adhering to standards such as the ISO’s] 27001 is the minimum viable product,” one CISO said. “I try to underline how we sell ourselves to our customers. We have a competitor who got popped, and they have a similar name to our business, so we have to be able to explain why we’re better than that. That changes the model. At the moment, [most companies seem] to be keeping the barrier at, ‘What’s the bare minimum we expect?’ We need to do better than that. And then we can change the model.”

Cowboys

Another set of tools that are widely used to help companies assess supply-chain risk are scorecards produced by companies who offer the chance to take up some of the burden of information-gathering for client firms, and put what is known about a supplier’s risk and compliance into a single, updated, easy-to-understand format. Most leaders in the room seemed to use scorecards, but few seemed particularly happy with what they were getting out of them. And the CISO at the firm who has a competitor with a similar business name was certainly not a fan.

   “We get called after a competitor gets breached, because we’ve got a similar name – and then we get scores that aren’t for us, because the [scorecard vendors] are following the wrong firm,” they said. “It puts us at risk. We have people who say that our score will affect our contract, but they’re following the wrong firm. We have to tell them who we are and what [standards and certifications] we’ve got, because if we don’t, they’ll get the wrong picture.”

“Do you not think you need to change the name of the business?” one wag asked, cheekily.

“We did!” came the resigned reply.

“This is an interesting point,” Dellow said. “People are trying to monitor their suppliers but they don’t necessarily have the right tools or resources to do it accurately.”

“They’re chasing the minimum,” one leader argued. “They’re using scorecards to monitor the supply chain, and getting it wrong.”

“I’m sorry,” another attendee responded, “but that’s their problem.”

Maybe, others acknowledged. However, as the security leader at the firm with the similarly named competitors noted: “It becomes ours.”

Dellow had some optimism to offer to this beleaguered CISO, and to any others who may find themselves in similar situations.

   “We help end to end, and we come across companies like yours all the time,” she said. “The business will go out, pop a domain name in [to the scorecard system], not even validate that it’s the correct one. That shouldn’t be down to you to resolve, but it comes up again and again. The questionnaire [responses represent] very much a fixed point in time, so these tools do provide something; there is some value. But, a), it doesn’t flag if it’s the correct domain, and b) doesn’t tell you how the score is relevant to the service that provider supplies to you. That relationship between you and the vendor is unique. If we rely too much on these scores – just like if we rely too much on the questionnaires – there will be problems.”

How Hard Is It?

There is clearly much dissatisfaction, both with the tried-and-tested traditional questionnaire-based method of gathering an informational baseline about supplier security and compliance, and with some of the additional tools and techniques that have been developed to try to help expand upon that. Clearly there is a widespread need for something that goes beyond these most prevalent of extant approaches. Diligent believe they have a compelling product set to help the many businesses clearly struggling in this area, but talking about it, their staffers said, is a necessary, and positive, first step.

   “We’re trying to solve those issues we’ve been circling round tonight,” Ryan said. “We built a platform that allows customers to monitor lots of different sources every minute, every day. Scorecards, but also the dark web, news forums – any place that could point to stress that would bring risk to your business. And then building a risk profile. We’re getting you to the point where you continuously monitor many, many different sets of data, and we help you proactively manage and mitigate risk.”

As to where these approaches may lead in the future, the discussion, Farrell said, had proved instructive.

   “One of the most interesting things said here was that AI has helped,” he said. “I’m very critical and dubious about it, but I liked how we heard that it has got other groups within the business involved. The importance of the rest of the business being involved in third-party risk management – of it not just being a security problem – is vital. You need interest from the board, and if it’s just a security problem you don’t get resources, so we end up with it being just questionnaires.”

“It’s clear no one approach works for everyone,” Dellow summed up. “We might all have similar roles, but we’re all in different businesses with different strategies. We’re all reliant on suppliers, and the questionnaires all look very different. And the risks are very different. We see more and more organisations looking to explore transparency on data provided, and how it’s validated. There are number of different ways of doing that but as we all grow and scale up it becomes more complex. Ultimately, you have to identify your biggest challenges, and how you can address them. We’d be more than happy to show you how we can help, but also to share what we’re hearing from other customers. We’re always happy to help connect the dots.”