THE FARCE OF INDUSTRY AWARDS AND PRODUCT BENCHMARKS
This article is cross posted to LinkedIn here for comments and discussions.
It's that time again. Before every big security conference, there is chatter between people deciding if they should or shouldn't attend. After it's over, there is chatter between people who did go, but this time about whether it was worth it. Sadly most of the time these days, the answer to the second question is no and a lot of people say “I told you so”. The general sentiment seems to be that many conferences which used to be iconic in the industry, are now flooded with more sales people than security people, talks have become thinly disguised sales pitches, vendor halls are full of tacky gimmicks vying for attention and security awards are a total joke.
Security awards are the focus of this article. At a recent talk, the Myths of Software Security, I opened up with some slides about them. Hopefully humorous, definitely sad.
I picked one company to illustrate the issue. I don’t know them from Adam. I could have equally picked on any one of hundreds. They are no worse than the others, they were just the first ones that showed up in my web search and were an easy target. I have blurred out their logo but it wouldn't be hard to find out who they are but it's really not about them specifically. They are guilty, they peddle lots of grand claims of magical security and it's on a public website, so they are fair game. Unless I used a real example it would just have been hearsay.
Like always, tucked inside a navigation menu, is an item for awards that takes you to a virtual trophy room page. Sometimes it is a proud hero image on the homepage that they have won something big that also steers you into the trophy room and sometimes a blog article, but no matter how you find yourself in their hallowed room, the awards always look and sound both important and impressive. The headline is usually accompanied by a bold claim and usually back-slapping about the company itself. This shows company momentum or this shows how amazing we are, nothing focused on how they have been recognised for solving an important industry problem. That’s because they haven't.
Here is an example.
“We are honoured and humbled by these recent awards from such prestigious authorities.” Well look at that. The curious can follow the award trail, in this case the award is for The Best DevSecOps Security Tool from The DevSecOps Excellence Awards. Sounds important. Sounds impressive.
When you go to The DevSecOps Excellence Awards, you will first see they are run by a marketing company. That should be your first red flag. It’s a marketing company that among other things organises awards. From their website “The Channel Company helps the IT Channel with communication, recruitment, enablement, engagement, demand generation, and intelligence.” That's right, they create “demand” and they do it by creating awards.
And it's a company where bow-ties seem to be in fashion no less, surely a second red flag?
From the home page of DevSecOpsExcellenceAwards.com you can see a menu item for winners packages. What, you get a prize? You get recognised and you get a prize? No.
Winners, if you can call them that as you will see below, can pay £2,749 (around $3,500 at the time of writing) for the privilege of getting your company logo next to the category in the awards listing. Really? Can you imagine when Brighton and Hove Albion eventually win the premiership, we have to pay for the trophy and pay to be listed as winners by the FA ? It’s clearly totally and utter madness, and totally and utterly worthless, but there is more. Let's look at who decides who is excellent and who is not.
I couldn't find anything about how the panel is selected but 5 out of the 13 work for the marketing company behind the awards, and I am not sure about you, but if I look at a panel of judges and see the head of content for the marketing company who runs the awards as a judge I am immediately going to call bullshit. While I am at it I am sure Terry is a stand-up guy but the Church of England is hardly an intuition I would look to for DevSecOps credibility. Then again he uses “security and digital anthropology in order to design agile, secure and modern workplaces and platforms”. WTF does that even mean? Have a word with the head of content will you Terry? I’ll give you an award for one of the most stupid, self grandiose descriptions ever.
Oh yeah and before I forget, Wiz, the fastest ever growing SaaS company in the world, lost the innovation sandbox to some company I had to look up. If that doesn't tell you something I don't know what does.
If you think the DevSecOps excellence awards are bad, enter the 2023 CyberSecurity Excellence Awards, another prestigious award the company in question boasts about having won.
Right there on the home page is what marketers call a CTA or Call To Action, telling you that you can sponsor the awards site or your own nomination. Hold up, you can sponsor your own nomination? Surely not.
For $1,900 you can, and this is in their own words “thereby significantly increasing your odds of an award win”. What the actual fuck. Is that for real ? Yes ladies and gentlemen, it is. You create custom categories that only you can really win. You get your logo promoted on the page where people vote. They post about you on their social media. But only if you pay them.
So this is obviously complete and utter bullshit I hear you say, no one falls for it, and it’s only the little guys desperate for attention that think these things have any value, right? No. Take a look in the right hand panel and you will see that VMWare is a sponsor and guess what.
VMWare Strikes Gold ! Fools gold for sure and look it’s twice in a row. Gold diggers ?
I could go on, and on and on. In my talk I also went through the case of the Cyber Security Almanac, touted as ‘100 facts, figures, statics and predictions’.
‘Facts’ from this like the ‘fact’ that 39% of security technologies used by companies are considered outdated, a statistic that is very convenient to a large vendor wanting to sell more of their stuff, happened to be quoted by the CEO of Cisco at last year's RSA keynote.
From boutique marketing firms turning a dime, desperate startups shouting for attention to the mega-corps of technology, this stuff has become pervasive. It needs to be called out by industry and more importantly by buyers. In an industry that should understand that trust is key, it's hard to believe that this is acceptable.
Be the change you seek right? As the person that runs marketing at Crash Override, I have decided we will refuse to accept awards that don't meet our moral standards and not play the deceitful marketing game of farcical security awards.
Footnote : Because it doesn’t warrant a separate post, the same is true of benchmarks. The same company above boasts that they are the market leader and the only tools that found all the vulnerabilities in the industry's benchmark.
Of course you did, the code for that test hasn’t been touched in almost a decade. That's when 4K TV’s started to come out and we had the iPhone 5 FFS. React born in 2013. I bet I could train a scanner to perform flawlessly on code that old with that amount of time.