Best In Breed? How A Global Pentest Army Can Help You Beat The Bug Bounty Blues
In Partnership With
It was a most un-RANT-like moment. Instead of debate, disagreement or dripping sarcasm, an opening gambit from a roundtable host organisation resulted in nods of agreement and a table full of rueful smiles. If there had been a microphone around, Synack sales director Dave Henderson would surely have insouciantly dropped it. Crowd wowed. Job done. Let’s go home.
Of course, the appreciatively stunned silence did not last long, though the security leaders in the room seemed roundly impressed with Synack’s pitch. The company offers businesses a managed, always-on penetration testing service, carried out by a hand-picked world-wide network of 1500 security-vetted white-hat hackers, promising a process more detailed, more accountable and more verifiable than can be obtained from the usual kind of penetration tests businesses carry out, or from running bug-bounty programs. Moreover, Henderson argued, Synack’s service doesn’t just find the problems – it helps businesses to fix them.
“Both traditional pentesting and bug-bounty approaches are focused on finding vulnerabilities that you then go and fix. Whereas we think nirvana – the place to be – is not just to find vulnerabilities, but to report,” he said. “Are we remediating? Is the remediation successful? How can you know if the remediation is successful? Do you understand how long that’ll take? Is it taking a long time because of a training issue or a resource issue?”
Instead of subjecting the concept to “ah, but…” critiques or “what if…?” theorising, the assembled expert audience instead immediately began to share their own experiences, concerns and thoughts on how and why the necessary but complicated business of realistic threat testing could best be accomplished.
Serve The Servants
The first item on the constantly evolving agenda for the evening turned out to be: what’s the point and the purpose of running pentests? This necessitated a discussion on what security staff believed pentesting was, and the security-focused reasons why it should be done – as opposed to what it ends up being when the parameters are set by the people who are paying the bills and have struck the deal with the test provider sit in a different part of the company and have entirely different motivations.
To security teams, penetration tests are a vital part of working out where problems may come from in future, and prioritising precious resources to ensure that the business has found all the doors, windows and other weak points, and closed and barred them to unauthorised parties. But to those who write the cheques, all too often, pentests are part of a compliance procedure, and are carried out to provide confirmation that security has done its job.
“We did a once-a-year pentest of the whole infrastructure,” one security leader recalled.
“We didn’t even know what that involved, because nobody could tell us. In the security team, we wanted the best pentest – but above us, they wanted it to give us a clean bill of health. It was just: test; re-test; show you have no critical vulnerabilities; and that was it.”
“There’s different ways of framing a pentest, too,” another attendee pointed out. “It succeeded because it found loads of critical vulnerabilities, or it failed because it didn’t.”
“Moving from box-ticking to real risk reduction only happens, usually, once there’s been a compromise,” one veteran security leader said, sagely.
“The real problem is prioritisation,” another suggested. “We have scanning running constantly in the background, and we do a once-a-year pentest. Functionality is provided and there are loads of reports. But how can we put the vulnerabilities that we’re finding into the context of the business?”
“And those generate very different backlogs,” a further leader pointed out. “Sometimes it’s great for your security team to have a big backlog and spend a year working through it; but your engineering teams might have a different timeframe.”
Help Me I’m Hungry
Additionally, attendees agreed, expecting to get anything useful from a penetration-test campaign would only really make sense if the business was consistently doing thorough vulnerability assessment and threat modelling, too. While elements of each of these disciplines overlap, each is necessary to support and inform the others. Yet few companies will be able to carry out all of them, and, even if they do, the results may prove, as an extended metaphor evoked by two leaders at the event suggested, somewhat indigestible for the business.
“It’s like having three different meals,” one said. “I’d like to have all three! But…”
“Some people will say breakfast is the most important meal; some will skip it; others can take it or leave it. It all comes down to the individual,” another added. “Not everyone has a devops cycle. Some will have engineering. Others will have an internet connection and block everything coming in, and everything else is SaaS [security as a service] and outsourced. You have to look at what’s worked for your organisation.”
This leader expanded their point, explaining that their business is effectively “a collective” involving numerous different semi-independent organisations. To over-extend the metaphor, those internal organisations will, effectively, be making their own decisions about breakfast, lunch and dinner; but the overarching organisation still is in charge of the overall nutritional balance.
“Each one, because it has a board, has a different risk profile,” they said. “As a group we have to set the bare minimum that’s necessary to secure the group. What you then do in addition is up to you.”
Endless, Nameless
Another problem raised, and apparently widely shared, was around the practical integration of the kind of testing that doesn’t just probe for weak points or attempts to find exploitable vulnerabilities, but which pushes network defenders into real-time responses. If a defensive team knows that a pentest is coming up, they will be on alert and half-expecting to see certain things: this therefore reduces the test’s usefulness because genuine adversaries are not going to make an appointment in advance. But if a network defence team were engaged in dealing with a genuine incident, the business would be taking a huge risk if it were to pile the pressure of defending a pentest on top of them at the same time.
Synack can handle this eventuality. The company calls its approach Penetration Testing as a Service (PTaaS), applying SaaS capabilities to what was previously a very manual process. Its testers all use a single company-developed platform, and their traffic will come from a narrow range of IPs, allowing for easy differentiation of a Synack test from a genuine adversarial attack, and also ensuring that Synack’s staff can, centrally and immediately, shut down the test if the customer makes that request. Traditional pentest providers, who essentially act as an agency for freelance white-hats using their own tools on their own IPs, will not be able to do this.
Similarly, many around the table said that their companies’ practice had generally been to rotate their pentest providers every couple of years, largely to guard against a growing familiarity with the company’s systems resulting in by-rote tests that delivered similar results each time. There are pluses and minuses to these kinds of policies and approaches, but, Synack suggested, their combination of a large pool of researchers and a robust project-management capability means that their customers can get the best of both worlds. To help emphasise this, one of their 1,500 white hats came to the event and was able to share some insight into what the work looks like from the sharp end.
“In a traditional pentest team, you don’t have hundreds of people – in my experience it’s been two, three, four,” they said, referencing a background that includes carrying out pentests for a variety of clients as a freelance researcher and on behalf of other pentest service providers. “What they tend to be is jack of many trades, less so master of some – though not all things apply in all cases. If you can pull from a bigger pool of people you get experts in niche areas – areas where it’s hard to find bugs, but they’re more impactful. By having that breadth of experts you’re getting, perhaps, what you would from a nation-state adversary. They’re going for something however long it takes – a week, a month, a year.”
For more on how Synack’s Penetration Testing as a Service platform can reduce the time it takes your organization to remediate critical vulnerabilities, visit www.synack.com. You can also connect with Dave Henderson on LinkedIn for more personalized insights on Synack’s approach to cyber risk reduction.
