Smashing Security podcast #457: How a cybersecurity boss framed his own employee

You know, look, you are fired, but at least you are in a world-class city where you have some extremely interesting tourist options at your fingertips.

Madame Tussauds though is very expensive, isn't it? And probably overrated. I'm not sure.

Listeners, write into Graham if you are as furious as I am with the suggestion that Madame Tussauds is overrated. Heresy.

I'm now going to get emails from the CISO of Madame Tussauds.

Smashing Security, Episode 457: How a Cybersecurity Boss Framed His Own Employee, with Graham Cluley and special guest Carl Miller. Hello, hello, and welcome to Smashing Security, Episode 457. My name's Graham Cluley, and I'm Carl Miller. Carl, welcome to the show, first time on Smashing Security. Fantastic to have you here.

I know, I can't believe it's taken you 457 episodes, Graham, to finally have me on.

It is rather outrageous, isn't it? I do apologise.

Yeah, you must have had some people on about 5 times by now.

Oh, some of them, oh my goodness, scores of times. I think that page on my Rolodex must have just, that's actually aged me, hasn't it, mentioning Rolodex? Must have just fallen out somehow or another. Now, Carl, for anyone who hasn't encountered you before, and more shame on them if that's the case, who are you and why might they have heard of you?

Well, I'm a technologist and a writer really. I always kind of gel those two things together. So I am a co-founder of an information integrity group of technologists, a lab I suppose you'd call it, called ChasmTech.

Right.

A think tanker. And people might have heard, I guess, in the cybersecurity world of me, if they have at all, via a podcast that went out at the end of 2024 called Kill List.

Now, Kill List was quite a sensation, wasn't it? I mean, it was a really popular podcast and it was about a fascinating topic, which we have touched upon in the past sometimes here on Smashing Security. Do you want to share a little bit more about what that was about?

Sure. Well, if the name hasn't completely given it away, it was really about a kill list. It was—

Ah!

Genius.

Yeah, I know. We like to describe things directly in the Kill List team. It's about an assassination market sitting on the darknet. So it was a long investigation that me and colleagues did. Essentially working with a hacker called Chris, who managed to gain access to the site, broke in, and found a way of intercepting all the orders being placed. And really, the whole series of Kill List thereafter is about what we have to do with all of that.

Right. And as a consequence, I believe people have been arrested who sort of ordered assassinations, didn't they?

That's right, yeah. It turns into this kind of global freewheeling, sometimes quite hair-raising, and often extremely surprising sort of investigation, which ends up us kind of working with the FBI and working with law enforcement agencies around the world. And I think we're up to about 28 or 29 convictions now, and just shy of 200 years of convicted time. You know, pretty serious criminality happening in the marketplace.

And one of my takeaways from all of these hire-a-hitman websites is you can't actually trust them, can you?

Well, I mean, you certainly can't trust their cybersecurity. I mean, it was laughably insecure, the websites, given what they were purporting to offer. You know, Chris the hacker found essentially that by cycling through the numbers at the top of the Onion address, he could bring up other people's orders. And so through that, essentially extract all the messages running through the site. And that was as good as handing it over to law enforcement. It wasn't, I strongly suspect, an actual law enforcement honeypot. But it was so insecure that it essentially allowed us to sit undetected in the middle of the site for years, essentially triaging them and gathering them, and then ultimately working with law enforcement to try and identify the people that were not just taking out, but also paying money to have someone killed.

The thing is that I'm not sure there's strong evidence if you ever did pay for one of these things online that it would actually occur. It's just that you are being scammed, a bit like you could be scammed by the offer of a free Apple Watch or something else, or handing over your phishing details. Equally, you could be scammed thinking you are buying an assassination. In fact, you're just putting cryptocurrency into the pocket of a common scammer.

Indeed. Yeah, the people running the site and the people running these sites in general, I don't think have any interest in actually delivering any of assassinations they're being paid to do. There have been a small number of documented assassinations that have originated from the darknet.

Okay.

Two, I think, one in Russia and one in Finland, to my knowledge. But they don't happen in these sites. They actually happen somewhat more chaotically or randomly, actually, in the margins of active, serious drug sites. So these sites, no.

Right. And you also do some work with a group called Demos, don't you? And what are they all about?

I do. So yes, I mean, I'm a very long-toothed think tanker, really. Think tanking, I know, is an odd field for most people. They probably wouldn't have heard of it or know what think tanks do, but they're essentially these little boisterous, noisy universities that try and do really specific research that tries to inform policymaking. And I've always been really drawn to this kind of envelope of technologically driven change in society and how government and policy and law and actually lots of institutions across society have to continue to adjust to that. So one of the big things that I have been fascinated by and I'm now working with Demos trying to do something about is digital democracy or essentially redesigning democratic processes fit for digital age and fit to be able to leverage and use emerging technology. So we've got this project called Waves. It's currently on the streets of Camden. It's going to go to South Staffordshire really soon, within about a month or so, to basically create new ways of people participating in political decision-making.

Hmm. Sounds interesting. Well, technology is moving at an extraordinary pace. I mean, I've just been reading in recent days some of the debacle regarding the Department of War over in the United States and their dealings with Anthropic and OpenAI and how AI could be used by the military and some of the goings-on over there. Do you have any opinions on that?

I mean, to say that we will be able to prevent cutting-edge warfighters from leveraging cutting-edge AI, I think is wishful thinking more than anything else. I mean, if there's any lessons that we can take from the battlefront in Ukraine, it's that every military has such a strong impetus now in pushing automations forwards as quickly as possible. I mean, I think there's two things that are really important. One is that we do that in a way in here in the UK and across our ally network that means that it's consistent and consonant with our own liberal democratic system of rights, but also that we do it actually in a way which is more innovative and faster than our adversary. I mean, liberal democracies cannot cede the technological playing field of warfighting to Russia or to China. I think that is really clear, Graham. And you know, we would wish it to be otherwise, but that's not the world that we are living in. So we actually need to become, as democracies, more innovative, not less, when it comes to how to fight wars. We just have to do that in multiple fronts, not just how to fight wars, but how to fight wars which are rights-respecting and casualty-minimizing.

It's quite the topic, isn't it? Well, let's move on with the rest of the show. And before we kick off, let's thank this week's wonderful sponsors, Meta, Action One, and Vanta. We'll be hearing about them later on in the podcast. This week on Smashing Security. We're not going to be talking about how AI can now link your anonymous posts to your real name. You'll hear no discussion of how a hacked Iranian prayer app sent surrender messages to worshippers amid Israeli and US missile strikes. And we won't even mention how streamers are worried that Amazon has just made wishlists a doxing risk. So Carl, what are you going to be talking about this week?

I'm, Graham, going to be talking about manipulation of LLMs. Now it's coming from a field known as FIMI, which I promise is the only clunky government acronym I'm going to throw at anyone today.

All right.

I don't promise. There might be more, but I'm going to keep them to an absolute minimum.

Okay. And I'm going to be talking about why you might not want to be invited to an offsite meeting. All this and much more coming up in this episode of Smashing Security. Smashing security. Well, we've got time now to chat about one of the sponsors of this week's show, Action1. And lucky old me, I've got someone here to help me do the ad. Say hello to everybody, Joe.

Hello everybody. Hello, Graham.

Hi, Joe. Now then, if you are a systems administrator managing endpoints every day, you've probably postponed patching at least once, not because you forgot, but because you didn't feel gambling with uptime.

Meanwhile, the backlog grows, vulnerabilities pile up, and patching stays stuck in manual mode.

Well, Action1 fixes that. Action1 is a cloud-native patch management platform for Windows, macOS, Linux, and third-party apps, all from one place. No VPN needed.

Curious on how easy it is to start with Action1? Well, you can use it on your first 200 endpoints for free, forever, with no functional limits.

First 200 endpoints for free forever. That's bonkers. Incredible, Joe.

So if you're looking to automate patching at scale and get weeks, even months of your time back, go to smashingsecurity.com/action1 and sign up for patching that just works.

That's right. It's not a disguised free trial. There's no credit card required. There's no hidden limits. All you have to do is visit smashingsecurity.com/action1 and get started today. And thanks to Action1, for supporting the show. And also, thank you, Joe, for helping me with the ad.

You're welcome.

Now, Carl, we've all heard stories about rogue insiders before, right? We've got disgruntled sysadmins planting logic bombs on the network. We've got contractors who copy the customer database onto USB sticks, often labeled definitely not the customer database. But this week, I'm going to be talking about something rather different. At the center of this story is a defense contractor called Trenchant. And Trenchant is interesting because it specializes in cybersecurity stuff, in particular exploiting zero-day vulnerabilities, the vulnerabilities that haven't yet been patched, that help governments and intelligence agencies and potentially, of course, cybercriminal gangs gain unauthorized access to systems. These are the kind of tools that governments will pay millions of dollars for. And, well, rumors were spreading around that a Western defense contractor might have suffered a massive leak that was exposing these zero-day exploits. They were thinking, is there a defense contractor who's got a bit of a leaky bottom and the vulnerabilities are falling out of them, and rather than being held securely, are falling into the hands of maybe nation states you wouldn't want them to fall into, or cybercriminal hackers, malicious hackers, and the like. So people were whispering about this, but they couldn't narrow down the details. They were effectively hunting for a bit of a ghost, a mysterious individual that they knew only as John Taylor. I mean, it's a fairly ordinary name, isn't it? John Taylor?

Very. Yeah, very much.

There's going to be a lot of them around. And this John Taylor was the name on the contracts which were being done with this operation called Operation Zero, a Russian outfit that rather politely claims to sell its exploits and zero-day vulnerabilities only to non-NATO countries. So they've worked out their niche. They say, we're not going to sell to NATO countries. That is not our market.

It's kind of like an opposite KYC, isn't it?

Yes.

We only sell to sanctioned entities.

That's it. But of course, John Taylor doesn't exist. At least this John Taylor doesn't exist. What we didn't know at the time was that he was just a pseudonym for someone else. So we shouldn't have been looking for a real John Taylor. It turns out it was someone who in reality, his colleagues knew as Doogie. And Doogie was this cyber espionage kingpin. It sounds like someone who'd be at home revising for his GCSE biology, but in fact, Doogie— oh, 'cause that's all I can think of is Doogie Howser. But anyway, Doogie Howser was a big deal. This Doogie, however, was a different Doogie. He had access to these vulnerabilities, these powerful exploits, which could maybe crack into Android and iPhones, cause all kinds of nuisance. Now, here's the thing. When the defense contractor Trenchant discovered that it had a leak, they didn't call the FBI immediately. They put one of their guys in charge of an investigation. They put a guy called Peter Williams, who headed up the US offices of Trenchant. He was known to his colleagues as— Carl, you're the investigative journalist. What do you suspect his nickname was?

It is not Doogie.

His nickname is Doogie. That's what they call him in the office. So the head of Trenchant USA, known by his mates as Doogie, was selling exploits on the side to the wrong sort of people. Operation Zero, the Russian exploit broker. And when Trenchant discovered that it had a leak, they didn't call the FBI. They put Peter Williams, the guy actually doing the leaking, in charge of the internal investigation. And of course, this guy, Peter Williams, he decides to save his own skin. So what he did was he handpicked a scapegoat on his team. He spotted that he had a talented iOS developer amongst his staff, and we're going to give them the nickname Jay Gibson. That's the pseudonym which has been given to this person in reports. Don't know what his real name is. In February last year, the Trenchant team, including this Jay Gibson guy, this innocent iOS developer, was sent to London for what Trenchant called a team-building exercise. In reality, it was an ambush because while the team was in London, Peter Williams, also known as Doogie, right? You're following this? Peter Williams, also known as Doogie, also sometimes known as John Taylor, appears on a video link from the States and with a straight face accused Gibson of being the leaker. So this iOS developer was fired on the spot while he was in London, his devices were seized, he was left stranded in a foreign country, branded a traitor, presumably left to make his own way home. And to twist the knife, shortly after he was fired, his personal iPhone received a message from the guys at Apple. And it gave him a notification that he had been targeted with some pretty serious state-sponsored spyware. You know how sometimes journalists and dissidents and people who generally aren't very popular with their government can receive messages saying, "You may well have been targeted by some serious spyware"? That's the message he got. Now, I don't know who tried to infect his iPhone with spyware or why, but it does make you think, doesn't it, since he had been working for a company which dealt with exploits, that he was someone who has been framed as the possible leaker of the exploits by the person who was actually doing it.

You know, you are fired, but at least you are in a world-class city where you have some extremely interesting tourist options at your fingertips and some great restaurants.

Yeah.

Yeah, I mean, for the past, I don't know, 10 years or more, we've all been worried about misinformation on the internet and we've seen the bots posting on Twitter and on Facebook and manipulating people's opinions that way.

Yeah, I think,

But what would be fascinating is if instead of individuals being targeted with this misinformation, if it's the AI being targeted and us humans go to the AI for advice and we get our opinions and we get our news and all the rest of it via this AI lens, we as individuals are going to be producing content as well, which has been poisoned somewhat if the AI has been poisoned.

I think a

So we'll be legitimate people who are spreading misinformation, perhaps unknowingly, because it's been fed to them by an AI which has been misinformed as well.

lot would be obvious.

Is that right?

Oh, well, I mean, that was talent shining through. I think we can all agree that there was absolutely nothing inorganic about the Cheeky Girls.

Why did they bring him to London?

I don't—

This is the really inexplicable part of the story. Was this just to make it more inconvenient for him to get sacked?

I don't know. I mean, maybe, right, if they really do believe that someone is leaking something, which the main guy in charge didn't believe because he knew who the real leaker was, maybe it was to get him away from his other devices. Maybe it was to get him away from his home network, or maybe it was to give them time to tip off the FBI, perhaps if they wanted to do that, so he could be caught upon his return to America. It certainly puts you in a more fragile position, doesn't it, being elsewhere?

Yeah, I suppose so. It's suddenly making me think, you know, never trust team building exercises.

Well, that's a general rule for life, Carl. Come on, for goodness' sake. We all know that.

I mean, I loathe team building exercises at the best of times, but now that I know that that can actually be part of an insider threat detection workflow, it's made me even less likely to sign up for one.

Okay. I'm thinking now, right? If a company was firing me, which isn't beyond the realms of possibility, to be perfectly honest, if they were employing me, maybe it makes sense to send me elsewhere in the world to make it more difficult for me to log in remotely and maybe cause damage inside the organization. Maybe they've got some kind of IP block when people connect to say, well, are you logging in from your usual IP address or are you suddenly logging in from London rather than Delaware or wherever it was you were? I don't know. I'm just speculating here, and I don't want to give anyone any ideas as to how they should fire me because I'm not the kind of chap who would do damage even if it was grossly unfair that you fired me. Let's stress that right now. You equally, Carl, obviously work for plenty of different people. You're not going to get up to any shenanigans, are you, if you're fired?

I mean, maybe it was to soften the blow.

Oh, well, that's kind.

Madame Tussauds though is very expensive, isn't it? And probably overrated, I'm not sure.

Listeners, write into Graham if you are as furious as I am with the suggestion that Madame Tussauds is overrated. Terrifying.

I'm now going to get emails from the CSO of Madame Tussauds.

Anyway, so just imagine you are being framed for a global espionage case. It's all the drama going on. While the real culprit is sitting in the boss's chair, watching your life collapse through a webcam on Microsoft Teams.

And he's called Doogie.

And he's called Doogie, which is the ultimate slap in the face with a rusty mackerel, isn't it?

It's very Kim Philby, isn't it? It's very Cambridge Five.

Now look who's dating themselves, Carl.

No, those were the days.

So, Peter Williams, the real head of the US branch of Trencent, he has now pleaded guilty, right? He was sentenced last week to 87 months in prison. They've uncovered that it was really him leaking these exploits to this Russia-linked exploit factory. Prosecutors estimate that his actions caused about $35 million worth of losses for Trencent. I wonder how much additionally they may have caused in terms of damages to other organizations who may have been hacked, whether it be by ransomware attackers or industrial espionage or spies, you know, all kinds of possibilities here. Because here's the thing, we still don't know precisely which zero days he sold to these Russian exploit brokers or where they then got sold on to. We know that they target Android and iOS, and we know that Operation Zero didn't keep them to themselves. They were selling to buyers across the Middle East and Asia. And we also know that some of these vulnerabilities had previously been used by hackers in South Korea, as well as also being sold to the Russians. So likely it's ended up in the hands of everybody from foreign spies to ransomware czars. The most unsettling thing I think about all of this isn't so much the technology, it's how ordinary it's become. These trusted insiders, it's about greed. It's very human, this. He even had a support contract with this Russian exploit broker as well, said, "Well, I'll help you if some of these vulnerabilities don't work properly."

It is an extraordinary story. I mean, what's really interesting as well is this kind of intersection between economically motivated cybercrime and geopolitics. Because, you know, it sounds like Doogie is in it for the money. But then for him to really be targeting sanctioned entities, you know, non-NATO countries, there's a whiff of the geopolitics in there as well, isn't it? It's not simply to the highest bidder. It's also to, kind of countries or actors that are lined up to confront the West. And it's often like that, isn't it, Graham, where you're not quite sure what is motivating anyone in these sorts of places, but where politics and money and geopolitical confrontation often are all swept up in ways which are quite hard to untangle?

Yeah, it may well be quite a cocktail. From my understanding, from reading of this, he was living something of the luxury lifestyle. So he was making plenty of cash out of doing this because, I mean, some of these exploits, you can sell for well over $1 million. So there is a lot of money to be made. I mean, everyone wants to crack into the latest iPhones and steal information, ideally with a zero-click exploit. But yeah, there may also have been something more geopolitical going on, or maybe there was some political motivation as well, or simply they just didn't give a damn and they just wanted an awful lot of cash.

For sure. I mean, and since time immemorial, the idea that a traitor or a turned spy turning up to the office in a Ferrari, you know, is often one of the ways in which they get caught. You can make our networks more sophisticated, but so often, you know, human greed and human error will finally catch people.

I'd just rather not turn up to the office in a Fiat Punto. And that was my big mistake. Well, we've just got a little bit of time now to chat about one of today's sponsors, Meta. So, you know how setting up a network for a new office location involves about 17 different headaches? You know, dealing with ISPs, designing floor plans, racking hardware, configuring everything.

It sounds like an absolute joy. I can't imagine the fun I'd have chasing an ISP for 3 weeks only to find they've spelt my company's name wrong on the contract.

Yeah, that's exactly what goes on, right? But Meta, what they do is they basically say, "What if none of that was your problem anymore?" Tell me more. So Meter is a network as a service company, but a proper end-to-end one. You give them a physical address and a floor plan. They sort out the ISP. They design and deploy the network. They turn up on site. They rack the hardware, which is their own hardware, by the way, not just reselling someone else's kit, and they get everything running.

So you get to skip the fun part where you're on hold with the telecoms company for 45 minutes.

Yeah, exactly. And then you get a single dashboard for monitoring, management, security, VLANs, firewall, DNS security, SD-WAN, the works. You keep visibility and control via their dashboard without having to do any of the tedious groundwork.

That sounds genuinely useful. What's the catch?

No nasty surprises, Joe. Just a straightforward subscription model. They even have a hardware buyback programme if you've already spent money on equipment from another vendor. Sounds cool.

Where do I find out more?

Just go to meter.com/smashing. Go and have a look there today.

That's meter.com/smashing. And thanks to Meter for supporting the show.

Carl, what's your story for us this week?

My story, Graham, is about large language models, AI, and FIMI. FIMI stands for Foreign Information Manipulation Interference. And it is essentially information warfare. It is this strange new kind of conflict which has opened up within information spaces. But its recent origin is recognition that militaries made in the mid-noughties where they reconceptualized information as being not just a tool of war, but in fact, a theater of war. Now, the specific story is towards the end of last year. It was in The Guardian, and it basically reports on this whole network of strange websites that counter-disinformation, counter-phoney researchers had found. They were hundreds of websites aggregating millions and millions and millions of individual news articles, all of which were basically about Russia or pro-Russian in some way. Lots were drawn from Russian state media and what was odd and what researchers were scratching their heads about was despite the eye-watering size of this huge pile of mainstream reporting. You know, it was on everything. There were stories about Ukraine and domestic stories about Russia, and there were things to do with Russian pop stars and there's quite a lot of culture and traditional values. It was not selective and it was certainly not filtered. But despite this huge aggregation of all this reportage across all these websites, it didn't seem, and it was a network, they called it the Pravda Network. It didn't seem this network was making any effort to make itself seen. There wasn't any real attempt to make it more visible, you know, to link it to target audiences. So there was one thing that we knew and one thing that we didn't know initially. What we did know, what researchers were concluding, was that this is part of information warfare, this is part of FIMI. Since the '00s, and certainly I think supercharged by the invasion of Ukraine, we have seen in information spaces states become much, much more active and muscular, threading and weaving together the tradecraft of cyber-offensive activity with the tradecraft of shadowy influence. Partly it's technical, partly it's cyber, partly it's to do with understandings about how influence works, how humans think, psychology and sociology. We've seen all these things weaving together to mean that states will now, certainly autocrats will now, systematically try and manipulate information spaces to have an effect, to change what we see, what we think, who we are, our sense of belonging and how we fit into society, loads of things. It was recognizably a FIMI campaign.

Yeah, because this isn't about hacking the AI directly, is it? This is about stuffing the internet full of rubbish or something which matches your particular agenda. This is about subtly trying to bend reality over time in people's minds by slowly seeding this misinformation into AI because people will now go to AI for so much. Their search engine effectively has now become the AI, hasn't it? That's how people find out things these days.

You're right. And I think that's a starting point realization, and that's why I wanted to foreground this story for you, which is that as soon as I saw this, I began to think it is absolutely inevitable that manipulating LLMs is going to become a whole new and perhaps the dominating new battlefront in information warfare. They're going to be the lens through which we see society. And not only that, LLMs are going to be some of the most precious and treasured relationships that many people have. This isn't just ChatGPT and Anthropic. We're talking about hundreds of therapy bots, boyfriend and girlfriend bots, pen pals, creative writing partners. People are going to create very deep, meaningful relationships with AI. And all of that is an attack surface. And I think that was the first in what is going to be an accumulating and intensifying conflict where the overall aims of information warfare to have influence begin to intersect with the technical realities of how do you manipulate LLMs in order to change their responses for people in short-term ways, long-term ways, tactically, strategically, fact-based, non-fact-based, changing how people feel emotionally. How they view the world as well as what they see. All of that is now up for grabs. And what you began to do, Graham, is exactly what I began to do, which is begin to ruminate on how this might happen. And the Pravda network probably wasn't that successful, but what it was probably doing was seeding the internet with vast amounts of text that it was hoping was going to get swept up in the training cycles. So for AI to begin to create new semantic patterns on the basis of all of it that will become more pro-Russian over time.

Yeah, of course. We, as we change, become agents ourselves of social change. And, you know, I mean, let's not sugarcoat this. It is conceivable that states developing sophisticated, scaled ways of covertly being able to manipulate and exert influence over LLM responses might be able to exact long-term, deep cultural changes on a target population. This is as serious a threat, not just to an election or an event or a particular moment, but to long-term senses of our culture and our ways of doing things as I think you can have in the information space. If I was a Russian information warfare officer, I would be doing nothing else. This is what I would be trying to develop and build ways of doing. And that means that LLMs, they're not stepping into this kind of neutral, you know, 1993, Google's just indexing the web. This is all so great. Hahaha, how fun environment. They're stepping into basically a battlefield. And I don't think anyone has realized that yet. And the battle over information integrity and then over our culture, society, integrity, polarization, basically everything. Part of that will now be a battle over securing these LLMs. And if there's one thing we've learned, Graham, from social media, which is what I've spent the last 10 years of my life or so, you know, researching trying to understand, this is not a battle that we can leave simply to the technology providers and platform providers to fight and win and be honest about by themselves.

But can we leave it to regulators? Can we leave it to politicians? They always feel they're about 10 years behind where the technological curve is as well.

Regulators, probably not. No, I don't know if there's anything in the Online Safety Act that will create an envelope for Ofcom to act. And I don't think Ofcom has acted fast enough when it's come to obviously and clearly illegal content. So no, I think regulators are already far behind. Politicians, I think, are now, via the bruising experience of trying to get social media companies to take responsibility and the bruising experience of seeing the last 10 years of warfare conflagrate across information spaces, I think are now more aware. So I'm not utterly defeatist, but no, of course there's a constituency that's going to be part of the solution that we haven't mentioned, and that is of course technologists. So we're going to have to build defensive barricades that can detect attempts to manipulate, can disrupt them. I think the future is essentially going to be different largely automated systems trying to find vulnerabilities to interdict and to manipulate LLMs, and similarly, other systems trying to spot those attempts and trying to defang them. That's the thing.

If someone did manage to poison an AI, would we even know that that happened? Would answers just drift slowly away from accepted norms? Would it be obvious something was wrong?

Because they're suddenly promoting Russian pop stars. I mean, I remember that band Tattooed. Yeah. They were unnaturally popular. Was that an early example?

Well, some people, some people suspect it was. Yeah. In fact, I spoke to The Economist who wrote an article that was in fact actually digging exactly into whether they were a Russian information operation.

Not just the Russian pop stars, I mean, there were the Cheeky Girls as well from Romania.

But, you know, you're talking about this Pravda network, obviously with the Russian links. If the Russians are doing it, surely everyone's doing this. I mean, is America doing this?

Yes, I mean, they already are. So the thing that really worries me is that there is a whole private sector industry called generative engine optimization, or GEO, that is basically now springing up. If you go to Web Summit, you go to any of these big startup global conferences as I do, you will now see a little section of one of the huge echoing exhibition halls basically crammed with GEO companies. They're taking VC, you know, they're beginning to grow and they're basically billing themselves as an honest and direct extension of the search engine optimization world. So what they're doing is they're building what they argue is just a new way of doing SEO in the age of LLMs. Yeah, when that conjoins with Russia, or China, or Iran, that is when we have a problem. Because you can SEO all you want if you are a brand. SEO, I think, has always been the weird, dirty secret of marketing, and no one's ever wanted to talk about it. But the reality is that loads of companies all over the world basically engage in quite serious scaled online manipulation in order to get visibility. But that is going to, I think, when bundled into an LLM, present us with a whole new batch of really quite serious problems, especially when it's being done for geopolitics or with more nefarious aims in mind.

Yeah, particularly as we see search engines being used less and less and more people using an AI. One of the things I do is I do a lot of keynote speaking, and I was on this bootcamp thing for speakers. One of their things they said to me was, well, are you optimizing yourself for LLMs? Are you writing content so the LLM will scoop it up and say, oh, Graham Cluley is a wonderful keynote speaker and very reasonably priced, you should go and hire him, you know, in particular industry sectors. And I thought, this is SEO 15, 20 years later. And now this will be the accepted wisdom as to what is truth because people trust these things so much. And as you said, people's relationship with the ChatGPTs and the Claudes of this world are very personal sometimes. You know, this is their friend they're speaking to or their advisor.

And you know, there's the ChatGPTs and there's the Claudes. And you know, I know some of the people that work in threat intelligence in these large foundational model companies. They're very clever. And I think the huge companies will build some kind of reasonable defensive response. Will it be enough? I don't know. But I'm far more worried about the kind of character AIs, spicy AIs of the world, right? There are hundreds and hundreds of foundational models that have now been built, many of them being open sourced. And there are thousands of companies that are taking these models, tweaking them, working on their own, you know, many of which see their route to success to be about creating a product which creates a much deeper, more emotional, more intense relationship between the AI and the human user. You know, you ask any sociologist how influence really works, and it's through your social ties. That is what really changes us. It's not randomly seeing something on the internet. It's who you are surrounded by and what they're like. We are so formed by social ties, Graham, that someone that you don't even know, that your friends know quit smoking, you're more likely to quit smoking. But that's how powerful this is. And it's also inevitable that our social network in the years ahead, for some people, and I know this sounds odd to some people listening to this, but people's social networks will include non-human AI models. We can see this. This is already the case. There are already people that are marrying their AI, and there's tons of people that are developing some kind of, to them, really meaningful relationship with AI. That is the attack surface I'm most worried of. If I was Russia, that is what I would target. And these do not have the hundreds of millions of capitalization that an OpenAI has to defend themselves. They actually have no real interest in looking for this as a problem because once it becomes a problem, then you have to do something about it and that will cost money.

Boy, you're depressing, Carl, aren't you?

Sorry, I have that effect on people. Once I gave a talk and afterwards the host came on stage and then called for calm.

Don't panic, don't panic. Yeah, exactly. Okay, before we go any further, Joe and I have got time to today, Vanta. So we've got a question for our listeners.

What do you worry about at 2 o'clock in the morning when it comes to your company's cybersecurity?

Is it, do you actually have the right controls in place? Is it, have our suppliers been hacked? Is it the truly terrifying one? Why are we still trying to do everything with spreadsheets, for goodness sake?

Well, if that sounds like you, enter Vanta. Vanta takes all that painful manual security work, like chasing all the evidence, filling out questionnaires, updating the same spreadsheet for the thousandth time, and it automates it. That's right.

Their trust management platform continuously monitors your systems, pulls everything into one place, and helps keep your security program audit ready all of the time. And yes, it uses AI, but in the useful way. Flagging risks, streamlining evidence collection, and fitting neatly into the tools you already use. All of this so you can move faster through the night.

Learn more and get started today at vanta.com/smashing.

That's vanta.com/smashing. And thanks to Vanta for supporting the show. And welcome back, and you join us at our favourite part of the show, the part of the show that we call Pick of the Week. Pick of the Week. Pick of the Week is the part of the show where everyone chooses something, could be a funny story, a book that they've read, a TV show, a movie, a record, a podcast, a website, or an app, whatever they wish. It doesn't have to be security related necessarily. Well, my Pick of the Week this week is not security related. There I was the other night with my lovely wife and we somehow got chatting about movies that had scared us as kids. And I played my top trump card. No, not that scary. My top card, which was The Incredible Shrinking Man. From 1957. Now, Carl, I'm not suggesting you watched it at the time. Neither did I. I wasn't around then. Have you ever heard of a movie called The Incredible Shrinking Man?

Oh, I had to watch it at the time to get over my shock of seeing the Cambridge Five being convicted.

It used to be on BBC Two all the time. It's a very simple premise. Ordinary chap, he's relaxing on his boat, and he's enveloped in a mysterious radioactive mist. And a few months later, he realized that his clothes are no longer fitting. They're getting a bit baggy and he's losing weight. And the doctors are baffled and they're doing tests. And he starts shrinking and shrinking and shrinking. And before you know it, he's a teenage boy and then he's just a little 9-year-old. Well, as he gets smaller, his life is collapsing around him. He loses his job. His marriage is under strain. Eventually, he's living in a doll's house. And is terrorised by the family cat. Very frightening.

I mean, we've all been there. I'm certainly terrorised by the family cat.

One thing leads to another, and he ends up in the basement, and his wife thinks that the cat has eaten him. And the rest of the film is really the story of how he tries to battle his way out of the cellar. There's a great big house spider. The effects, by the way, in this movie, for 1957, for black and white 1957 movie, are tremendous. Everyday objects are towering over him. Thread becomes climbing rope. There are drips of water threatening to drown him. I mean, this is 70 years old, this movie, and it really scared the wits out of me when I was a kid. And so I wondered, is it still going to work? So I sat my wife down and said, look, we're going to find this. I tried to find it online and I eventually found it on the Internet Web Archive. So I'll put a link in the show notes if you want to watch it. I think it pretty much stands up. I think it's still a good movie. The ending though wasn't really the best to my mind. I'm not going to spoil it for you. After all, it is a 70-year-old movie. Some of you may not have got around to watching it yet, but I would recommend The Incredible Shrinking Man if you're in the mood for classic sci-fi. It's tense, it's inventive, extraordinary effects for the time. So give The Incredible Shrinking Man a watch as it is my pick of the week. Link in the show notes. Carl, what's your pick of the week?

Well, I was worried before coming on that picking a book would be kind of too keen and earnest, but I have picked a book. It's a brilliant book and it would be a shame not to take up the opportunity to mention it to everyone. So it's The Immortalists by Alex Kratosky. She's a brilliant technology reporter and writer and in fact, podcaster and broadcaster. And she's just written a new book basically looking at the contemporary human attempts to try and defeat the ravages of time and that mortality itself. And of course, it's not just generally that people are doing this, but in fact, one particular community that you might be able to guess, it's the technologists of Silicon Valley. And so they've reconceived of the body as an engineering problem and cellular decay as an information theory problem. And it follows different tribes, different groups that are attempting to do this, which involve grifters and strangely anti-science, almost anti-vaxxer groups. And certainly people that are preying on other people's vulnerabilities, but then also individuals that are genuinely now beginning to demonstrate that they are slowing down the process of aging according to a series of objectively measured biomarkers. There's one individual in the book that Alex thinks might actually have genuinely stopped ageing. I think for almost everyone, we think that death and ageing is an inevitability, and to resist that is a heresy. And to know that there is a group of people out there, the transhumanists and the long-termists who fundamentally reject that, that would see working on the problem of ageing to be right up there with Mars colonisation and the other preoccupations of the Valley, I think is, to me, completely fascinating. I've never really seen bio-science and bio-engineering to go hand in hand with the PayPal mafia and crypto, but they do. And it was totally engrossing to me. And Alex is a brilliant writer. So, that's my pick of the week, Graham, The Immortalists.

Now you mentioned Mars colonization, and when you started talking about this, first person I thought of was, of course, Elon Musk. And it does feel like he would rather like to be immortal.

He is certainly mentioned in the book, as is Peter Thiel and a series of other Silicon Valley billionaires, and a process called plasmapheresis, which is basically pumping young blood into an older body. I mean, if there's an image that I might leave people with, let it be this. One of the lab experiments that really kick off is where they literally stitch an old mouse to a young mouse. And in doing so, and this is apparently a thing that you can do in biological experimentation, the blood systems fuse together. And as they fuse together, of course, you've got young blood being pumped into the older mouse. And you can see the degeneration of the organs begin to slow down, as in the actual organs in the older mouse get younger. And that's what really spurs plasmapheresis forward. So there we go. Two mice sutured together to form one mouse. Good evening, everyone.

I was about to say, yeah, Carl, do people invite you to parties very often?

The invites are dropping off, Graham. I'm not going to lie to you. The more that I talk about the mice being sutured together, the less London seems to want me to be there.

Well, that just about wraps up the show for this week. Thank you so much, Carl, for joining us. I'm sure lots of our listeners would love to find out what you're up to, follow you online and all the rest of that. What's the best way for them to do it?

Hit me up on LinkedIn, everyone. Yeah, yeah. And send me a message. And if anyone wants to talk about LLM manipulation, they think it's happening, they've got an idea of how it might happen, it's happened to them, they want to confront it, let me know, because I want to really do something about that over the next year. Fantastic. And I'm up on LinkedIn as well, Graham Cluley, and you can follow Smashing Security on Reddit, Mastodon, and Bluesky as well. And don't forget to ensure you never miss another episode.