RBS, Rapport and OITC anti-virus test results

An email from a customer today brought my attention to some anti-virus test results that have been published on the website of RBS (Royal Bank of Scotland).

At first glance, the test results look quite bad for Sophos (and even worse for Symantec and McAfee).

OITC test results promoted on RBS website

But if you dig a little deeper into the methodology used by OITC to come up with the results – published by RBS on their page promoting a security add-on called Rapport – then you actually find that the methodology is flawed, and that these test scores are about as useful as a chocolate teapot.

Sign up to our free newsletter.
Security news, advice, and tips.

As Stuart Taylor describes in a post on the SophosLabs blog, OITC’s methodology actually penalises the likes of Sophos for their ability to proactively detect brand new malware using (in our case) behavioral genotype protection. That’s because they exclude from their tests any piece of malware which they find 25% or more of security products already detect.

That’s bonkers. (They did this I presume in the mistaken hope of determining if a piece of malware was new or not, but in the process penalised products which proactively detected it).

Furthermore, these results don’t give any allowance for layers of protection such as run-time suspicious activity or buffer overflow detection, both of which would be defending customers in the real world.

My advice? Check out the independent comparative tests from the likes of AV-Test, AV-Comparatives and Virus Bulletin. They may not always put Sophos top of the class for virus detection, but I sure trust their testing methodology more than OITC.

I hope in the future RBS might link to some of those tests for a more helpful indicator of the performance of anti-virus products in the future.


Graham Cluley is an award-winning keynote speaker who has given presentations around the world about cybersecurity, hackers, and online privacy. A veteran of the computer security industry since the early 1990s, he wrote the first ever version of Dr Solomon's Anti-Virus Toolkit for Windows, makes regular media appearances, and is the co-host of the popular "The AI Fix" and "Smashing Security" podcasts. Follow him on Bluesky and Mastodon, or drop him an email.

What do you think? Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.