Impatient Leaders And Troublesome Priests: Why Security Leaders Really Worry About AI

It was billed as a conversation about how, notwithstanding the pace of adoption of so-called “AI” systems, the fundamentals of cybersecurity haven’t changed all that much. So those of us attending a RANT roundtable in London, sponsored by Cisco, had perhaps been expecting a few time-served examples, war stories or talking points that dug fairly deeply into the past to emerge during the evening. But we were expecting that past to be rather more recent than turned out to be the case.

One veteran security leader at the table thought it was important to go back beyond not just the growth of cloud computing, the work-from-home revolution kick-started by COVID, or even the birth of digital networks entirely. No: there are things that haven’t changed since at least the year 1170, when one senior executive who viewed their job as being to warn those at the top of the enterprise of when too much risk was being accepted paid the ultimate price.

“In Canterbury Cathedral there’s a shrine to Thomas Becket,” our beleaguered CISO friend said, initially to some bemusement around the room. “He told the king – his C-suite – that something wasn’t a good thing to do. Then the king said, ‘Can someone please get rid of him’? so a bunch of knights martyred him. And that’s what’s happening to security people.”

Tellingly, while many around the table smiled, perhaps in recognition that the analogy was rather extreme – after all, we’ve not heard of any security leaders being hacked to death on the orders of their CEOs (well, not yet, anyway) – nobody took issue with the basic truth outlined. When it comes to generative AI, the kings of the business – the executives, the board, the elites at the top of the organisation – are gung-ho for these new tools to be deployed within the organisation, so place implied or sometimes explicit pressure on the senior leadership who report to them to get things moving, and fast. But when the security specialists point out the risks involved, and advocate for taking time to get the deployment right so they can ensure that the business can remain as secure as possible – or, failing that, to at least be demonstrably resilient when the eventual attacks hit – the kings just don’t want to know.

“The C-suite are saying, ‘I’ve read about all this in the FT or Forbes’,” our student of the medieval world continued. “It’s FOMO,” they added, demonstrating their linguistic and conceptual agility by switching from 12th century history to 21st century vernacular in a heartbeat. “A huge amount of FOMO from executives. There are senior members of management who are going backwards.”

When Will We Learn

This contribution came nearer the end of the discussion than the start, but – despite how striking and unexpected the imagery may have been – it tapped in to one of the key themes of the evening. That was that the pace of adoption of AI is not being matched by growth in maturity of organisations when it comes to understanding and managing the risks that potentially transformative new technologies introduce. And, while nobody in the room seemed to have made a conscious decision to pile in on Microsoft, a lot of this part of the discussion came out in the form of complaints about the software giant’s chatbot, Copilot.

“I have friends,” one leader with a particular animus against this particular Redmond product recalled, “who say that Microsoft gives you access to the Foot Gun – Copilot; then they give you a bulletproof shield, called Purview, to stop yourself shooting yourself in the foot.”

It is, many attendees acknowledged, a powerful tool. “Prior to Copilot, finding information was difficult – but now, if you want to find something on your own corporate environment, Copilot will find it,” one leader said. But, many also agreed, it will find things that, on balance, you would probably prefer that no tool could.

“We’ve enabled Copilot for corporate access – [it can access] Sharepoint, emails and so on,” one senior security leader said. “But we realised that, in Sharepoint, it’ll have access to…” They paused, working out how best to explain the situation.

“My boss, the CTO, asked Copilot, ‘What’s the salary for everyone in the C-suite?’, and they got it,” they said. “We’re now looking at a technology where you create a digital twin – you get your own personal assistant living in the cloud. It copies all the documents you’ve access to in Sharepoint. We can put in restrictions on Sharepoint , but the twin can bypass them. It takes one copy of everything you’ve got access to. We all know that the attacker just has to be right once, and we have to be right all the time – but we now have to be right all the time on multiple fronts. If it was easy we’d do it ourselves and there’d be no risk – but the balance of power has shifted.”

Profits Paradise

Optimism has been expressed that generative AI will help defenders, and to a degree this sentiment was shared by attendees during the discussion – despite the view expressed by one CISO that AI “is like a four-year-old child: all it wants to do is please”. But the focus was very much on the risks that these technologies are adding to the enterprise. And, in large part, these risks are mounting because of the pressure being exerted by business leaders on the rest of the staff to leverage the productivity gains and work-speed improvements LLMs appear to offer.

“I’m hearing you say that Copilot is the problem,” Cisco’s global security technologist, Ant Ducker, said. “But we’re also being asked to be creative with AI. Is that the problem? [Business leaders say] ‘Here’s Copilot – we’re not going to give you any definitions, we’re asking you to figure out how to use it.’ Shouldn’t the business be saying, ‘Here are the things you should be looking to use it for, to increase productivity’?”

“We’re playful animals, and we learn by playing,” one security leader replied. “You don’t read documentation or worry about obeying rules – you just play with it.”

Fortunately, there are a few businesses where limits are imposed amid what otherwise appears to be a headlong dash toward AI adoption. But even in those organisations, security leaders are being put under pressure to do more, and do it faster.

“We’ve got a very well-defined process for cloud services and new emergent technologies,” one CISO said. “Our average time from the business saying ‘I want to use this new service’ to getting something in production is probably a couple of weeks. But for anything involving AI…? Copilot took us 18 months. We needed to put a harness around the harness – we have to put controls around it, and figure out how to make sure that all the regulations and expectations are met if we’re going to let it into the wild and have our population use it.”

It’s A Gamble

This talk of an additional harness raised some questions around the guardrails supposedly built into Copilot, and other LLMs, and to what extent they are effective or reliable (general consensus: not very). All of this means that internal policies and controls become ever more vital – as does having a maturity within the organisation when it comes to considering risk.

“We all seek to gain advantage,” one security leader said. “We’re all risk advisors. And certainly, in my experience, that means a whole host of different risks, including risk to life. We do a layered approach: it’s not risk removal, it’s risk reduction. And this is down to the CEO. This is what we need to realise – what and who we are. We’re risk advisors in a risk environment, and what we do is risk reduction, not risk removal.”

There was agreement with this point of view, but also some additional nuance that another leader wanted to inject into the conversation. Most risks, they argued, could be mitigated with some element of care around introduction of the new product, service or tool. The additional risk with generative AI tools seems to come, they argued, from the pace at which business leaders want to introduce them, and the circumvention of normal processes that meeting these aggressive timetables requires.

“I’ve worked in a hazardous environment,” they said, “and when you’re working in an environment where there’s extremely high risk, the idea that you’d go along with vibecoding, or would say ‘Well, there’s going to be vulnerabilities, we might as well just go with it’…” They stopped and shook their head at the sheer folly of such a notion. “No, that’s a really bad idea. You need to choose an environment – sandboxing or whatever – where you have an ability to control things and test things.

“This is the worst thing about AI being pushed in so fast,” they continued. “Dev environments have been around a long time, but at the moment they’re being short-cut. Things go straight into production.”

Heavy Mental

Ultimately, everyone seemed to agree, the only thing that’s changed thanks to LLMs is the pace with which everything happens. That covers not just the alacrity that senior corporate management seems to have for deploying the technology, but the speed with which it can wreak havoc in businesses that have failed to prepare for its arrival.

“I deal with simulation – redteaming, threat intel,” another senior practitioner said. Throughout their time in this role, they pointed out, “none of that has ever touched a vulnerability – it’s always touched a human. Can I find the human who can get me in to whatever it is I’m trying to get? With AI, now we’re going at speed. We all need to be cognisant. In organisations we’re going to see a lot of collateral damage. As people who convey risk, we need to convey it in a balanced way.”

“We need a central management pane – one pane of glass to manage everything,” Cisco’s Ducker suggested. “And in that place, that’s where we use AI for good. We create an army of agents that are network security specialists, identity specialists: we can monitor what’s happening across all the domains in our infrastructure, and we can collaborate. Rather than having four teams using their own UI [user interface], they’re all running from a dynamically generated UI.”

These capabilities, Ducker said – almost apologetically, as, he stressed, the company were not hosting the conversation as an opportunity to push a product or service, but to hear from senior practitioners about the challenges they were facing and how they were tackling them – fall within the bounds of the Hybrid Mesh Firewall concept that Cisco have adopted. A term coined in 2024 by Gartner, it “describes a central management pane that can manage a consistent security policy across multiple platforms,” he added. Cisco’s implementation of it goes further, “using the network as security fabric, and blending different kinds of security and enforcement capabilities right across the stack.”

“Listening to this, we still get back to – if you get the fundamentals right, you’re in a really, really good position,” one of the attendees said. “That hasn’t changed since the Orange Book,” they added, referring to the U.S. Department of Defense’s Trusted Computer System Evaluation Criteria standard, published in 1983. “Although it does look scary, I’m starting to think more and more – what does it change?”