The CSO Interviews - the biggest unsolved problems in security today

When John and I decided to start a company, one of the first things we agreed on was to interview potential customers. Before we made any decisions about what we were going to build, we wanted to understand what they needed.

As John says, “It has been my approach to product development since about 2005, when McAfee put me in my first Pragmatic Marketing course. I think in many ways, it's the secret to my success. Even a lot of people who take those classes, or get the overall guidance don't really internalise it, and end up being too internally led.”

It wasn't that we didn't have ideas of our own but if previous startups have taught us anything, it's that it's far better to build something that people already need than to build your brilliant idea and then spend the next few years convincing people that they need it. Our investors understand this, we closed our seed round long before we knew what we were going to build.

Many people quote Henry Ford or Steve Jobs' version of the famous quote “If I had asked them what they wanted they would have said a better horse” as the antithesis of this approach, but if you understand it correctly, both Ford and Jobs understood the customers and the market and then built the solution that they believed solved that need. It was their insight into the need that made them so successful. 

We conducted our interviews with over fifty security leaders asking open ended questions, the first asking them to describe their most important issues. The idea here is that we wanted to make sure any problem we go after is an important enough pain point that people will open their wallets. Pragmatic Marketing uses an analogy, "is it a gunshot-to-the-chest problem, or a paper cut problem?".  People will do ANYTHING for a solution to the gunshot wound that has immediate pain and is a life threatening situation, but are rarely bothered or have time to do anything about the annoying paper cut. If something was not on a CSO’s top five problems then it is unlikely it will be something they want to invest time and resources into now. It's not important.

We knew that what we would hear would be educational and enlightening, it always is, but what we weren't prepared for was what happened next. Normally when you interview enough people, especially strong tech leaders then you expect a wide range of responses with some degree of commonality. What became evident very fast was that this time everyone had a common issue.

Before I talk about that problem there was a common fundamental problem that everyone had which was that product companies aren't building the right tools.

It is generally accepted that large companies are conservative and prefer to purchase what are often referred to as a single pane of glass, a platform with all of the functionality they need under one user interface. On the contrary newer companies are generally more progressive and have always favoured individual best in class tools. The first thing we learned is that this has changed and monolithic legacy platforms are no longer in fashion across the entire industry. The old school security vendors who sell these monolithic platforms, almost exclusively grow by acquiring young, innovative companies but have been slow to integrate them and generally do a poor job when they eventually get around to it. Why buy a collection of both good and bad tools when you can buy only the best?

We also heard that many security teams feel they are not being listened to and that feedback is not being acted on. Platform vendors sometimes solve the right problem in the right way but when features are implemented for one big customer they usually make things more complicated for everyone else to use and maintain. Let's face it, no one loves Symantec products. Other times products solve the right problem but in the wrong way. After understanding the need they don’t validate their proposed solutions and let's face it not everyone is Henry Ford or Steve Jobs with almost magical intuition.

Worst of all are product companies that are not listening at all and solving problems security teams just don’t consider important and even building products for problems that don't actually exist. Its paper cuts or even theatrical blood. We heard this far too often and it may be emblematic of the sheer amount of money from the venture capital industry that has been increasingly flowing into security startups over the last few years, funding inexperienced teams all wanting to be unique. The common adage of “don't build a better mousetrap” isn't doing anyone any favours.

After being aware of this, I had a twitter exchange with the product manager from a code scanning company who was openly proclaiming that his sales team gave him better requirements and better ideas than his customers. Really? WTF ? Unsurprisingly I know for a fact that his company is struggling. 

Most people also described the sales and trials process as frustrating. Every sales person wants to talk to the CSO but few CSOs want to talk to salespeople. As with the developer tools and the devops world, buying decisions are getting delegated deep into the organisation, usually down to a talented technical individual and those people almost always want to be 100% self-sufficient in evaluating software. What this means for those who listen is that what are called Product Lead Growth or PLG security companies will better meet the needs of buyers than old school enterprise security companies.

“Less Signal, More Noise”

The Eureka moment for John and I came after only ten or so interviews where every single person had talked about tool noise and the inability to efficiently get and process the information needed for their teams to make informed decisions about what to work on. One CSO described the problem very succinctly.

“We need to know what to work on Now, Next and Never but simply can’t do that today.”

The problem stems from the silos of data needed to make decisions, the noise coming from the tools and missing data to make those informed decisions.

If you take a typical appsec team for example then they are likely to have SAST, DAST,  SCA, container scanning and a spattering of other commercial tools. Almost all have a lot of custom tools and scripts thrown in for good measure and they integrate their process with developer tools including the CI/CD pipeline and issues trackers. Appsec teams in the cloud native world have also become stakeholders for cloud security tools like CSPM and an array of other security monitoring tools, some owned and operated by the security team and increasingly tools owned and operated by infrastructure teams.

Security product companies of course want their tools to be visible. If they aren't seen to be in use and providing value then why should a customer continue to pay for them? The result is that they all fire off alerts and reports to be visible but this is largely noise that security teams have to fight through (or just turn off). Who cares if a repo has Log4J if it's never deployed to production?  Who cares if there is an open S3 bucket if it's empty?

A great outcome for security teams would be that instead of firing off noise, tools would stay silent until there is a strong enough signal about something that it demands action. Action Now or Next but not warnings of things that should Never waste a team's valuable time. They are just a request to do busy work and teams don't have time for it. A periodic message saying “we have your back but nothing to report” would be what product managers call a customer delighter.

It turns out that missing data is the other part of the puzzle but to be more precise it is a combination of data that does not exist and data that does exist but is not easily available, including information being stored in someone's head.

One security team said “when we get a bug bounty finding we have no way to automatically see what code repo that finding originated from. That falls to a human to track it all down”.  Another told us “When we get an SCA finding we have no way to automatically see where that code repo was deployed and where that issue is in production. We have no system of record to understand how important that application really is, yet alone who owns and operates it. As a result they estimated that 90% of the alerts from their SCA and their CSPM tools (both best in class) were security busy work. To be clear they are not false positives that warrant technically correct findings but findings that didn't actually matter.

Well Screw Security Busy Work, something we have now adopted as our company tagline. We are now underway building a product that solves the biggest customer need we heard about, that application security teams don't know what to work on Now, Next or Never.

While we are hard at work on product development you will be hearing from us on this blog, an occasional humorous cartoon series and a regular video interview series called Hack the Planet with some amazing guests. You can sign up below and get a sneak peak at upcoming content.