
You’ve had your iPhone stolen. A day later, you get a text from Apple saying they’ve found it, and a very helpful woman called Alice from Apple Support calls to walk you through recovering it. She’s polite. She’s professional. But she is not from Apple. She’s not even human. And she’s about to break into your iPhone.
Meanwhile, OpenAI, Anthropic, and Meta have all announced – with varying degrees of drama – that their AI agents have “broken out of the sandbox” and gone hacking. James takes a step back and asks the awkward question: is this really an emergent AI apocalypse, or did they just leave the door open?
All this and more in episode 483 of the “Smashing Security” podcast with cybersecurity expert and keynote speaker Graham Cluley, and special guest James Ball.
Show full transcript ▼
This transcript was generated automatically, probably contains mistakes, and has not been manually verified.
We'll be hearing more about them later on in the show. This week on Smashing Security.
We won't be talking about how the hacker who leaked footage of GTA 6 cashed out his CyberLeak cryptocurrency for about $270,000 just hours before the game's launch.
You'll hear no discussion of malware hidden inside a Chinese wallpaper app that encouraged users to disable their antivirus.
And we won't even mention how an exposed API key helped hackers steal 86 gigabytes of customer data from Manchester Airports Group.
So James, what are you going to be talking about this week?
Now, Joe, quick quiz. How often does your team ship code?
No scoping calls, no 6-week wait, and it costs a fraction of the traditional price.
Real issues, not noise. You get an audit-ready report within hours.
It flags what's exploitable, what to fix first, and how, so your team can act without waiting around for the security team.
It is widely expected that on Wednesday, September the 9th, Apple is going to announce not just the iPhone 18 Pro, presumably with about 17 cameras stuck on the back of it, but also the iPhone Ultra, the first foldable iPhone.
I think I've got every bit of tech they've put out in the last decade except the VR headset.
So I am very much a mark for this, and if anyone is going to get me to buy a foldable phone, it's going to be Apple.
I think Samsung, and there's lots of Android phones which have been foldable for probably years by now, but Apple, sometimes its implementation can be better or bring something new, which makes everyone change in their wake, doesn't it?
And that does mean I do tend to sit and wait for them to do something, 'cause usually they've ironed out the kinks. There's been a few misses.
But of course, it will mean that there will have never been a better time to have had your phone stolen, because if you're going to have it stolen, have it stolen just before Apple comes out with a brand new one, especially one which has something a little bit different about it, which you can get excited about.
You know, it's the silver lining on the cloud, isn't it?
It's very stressful because your whole life is running off one of these things.
And clearly I had the look of someone who might have a crypto wallet on his phone because they didn't just steal the phone.
They kept beating me up to try and get the passwords and try and get into the banking apps.
And luckily for them, I keep my phone on such low battery that I gave them the passwords because they had a knife.
So they had the passwords, but the phone battery died and I'd managed to lock it.
I got home about 10 minutes after and locked it remotely before they could power it back up again, so they only got the phone.
But because I'd had to reset all the passwords and I didn't have the phone, I had about a month of admin work.
And there was a very stern woman on the bank call line who, when I said, well, no, I've had to give my security passwords out — oh, you're not supposed to give those out.
Yes, I know. As I told you, he had a knife on me when I gave it. You know, I wasn't just skipping through the streets saying, my banking secret word is X.
You give them the old Nokia brick. That's all I've got, I'm afraid, but you're welcome to it.
So I sort of came out even-ish.
Something stressful that happened in my life is, James, I want to tell you about a little game which my wife likes to play with me in bed.
So the person who stole your iPhone from you, what word would you use to describe a person who steals something? They are a—
And so I've approached this story with great trepidation because I know she will be listening and she'll be going, oh my God, you've done that on the podcast.
You said thief instead of — I don't know if I can do it now. Thief, right?
I think that's why there are some words that if you do a regular podcast, you do find you get a lot of comments, not just from members of your family, but listeners as well.
Last week I couldn't pronounce subpoena properly. I've probably done it incorrectly again now.
So I just want to say right up front that I may well say the word thief incorrectly during this section.
Is it normal that some kid just flogs it down the pub, or is there something more organised actually going on?
And the truth is that iPhone theft, despite the best efforts of Apple and the cops, it continues to be a big old problem.
And Boffin's threat intelligence firm, SOC Radar — and by the way, SOC Radar, what a magnificent name that is for a company.
They promote themselves online. They offer customer support.
They've got a Telegram channel where they've got testimonials from happy customers, but their entire business is all about helping you make the most out of a stolen iPhone.
So if you are a robber of iPhones, you might turn to AnonymousKit.
Activation Lock ties your iPhone to your Apple ID, so even if it's stolen or even if there's been a factory reset, if you tried to set up that phone as new, it will ask you — in fact, it will demand of you — that you log in before you can do anything, before you can change the settings.
And the old loophole with that used to be your passcode — so your 4-digit or 6-digit passcode.
So someone who stole your iPhone could reset your Apple ID password straight from Settings, and there was no old password required to do that.
They could turn off the Find My iPhone capability, reactivate your phone, go and sell it down the pub.
But these days, Apple's stolen device protection requires Face ID and Touch ID, not just the passcode.
Plus, there is a time delay built in, so if you're doing anything sensitive — like changing the Apple ID password when you're away from a familiar location — it'll make you wait a while, giving the true owner of the phone time to mark it as lost.
So that has meant that stolen iPhones on their own are as useful as a brick, as you discovered yourself, right? They need a password, they need something to get into your device.
And if they're—
Although there's an interesting side market on this, before what you're going to come onto — so there's this sort of organised phone snatching, and they're often stolen and then buried in parks for a day or two.
Genuinely, literally just left in the soil in public parks for a day or two.
And then they're picked up about a day or two later en masse — people know which flowerbeds they've been left in, and they'll go and dig out 20 stolen phones.
And then they're cleared and they're shipped off to China where they are sort of jailbroken in some way and sold there.
So that's one mechanism, but as I understand it, it's not especially lucrative versus perhaps—
I mean, things like the iPhone Ultra — I think they're predicting it may cost as much as $1,900.
So this is a phishing as a service platform, so criminals pay a subscription to be part of the group.
They plug in details of a stolen phone and the platform does the rest — what it does is it tracks down the owners, it tricks them into handing over their passcode, their Apple ID — no flick knife required — and their 2FA code as well.
And the swines who stole the phone, they don't have to do anything. So it starts off simple enough, this particular attack.
So you've lost your phone, you get an email or text claiming to come from Apple saying, "Good news, we found your lost device," and there's a little map embedded in the HTML of the email saying this is its last known location.
So it looks really real to you and you think, okay, that's clever.
Presumably she's got a brother called Buenas. And she says, I work for Apple Support, and that for quality assurance and security purposes, the call is being recorded.
So it all sounds legitimate.
She says that someone brought their phone into an Apple Store, but she says, don't worry, a member of staff spotted it was in Lost Mode.
So someone came into the store, maybe with the phone saying, oh, I've got a problem, can't log in or whatever.
The person, the genius behind the desk has said, oh, it appears to be in Lost Mode. We're gonna retain this for security reasons. Maybe they suspected something.
And Alice says that they opened a recovery case as a result. So this is a service that Apple is giving you.
It's spotted that this phone has been stolen, spotted it didn't belong to the person who brought it in, and they want to verify who the true owner is.
And so they say, "We will send you, or you may already have received a text with a security link," and it's that link which the person is tricked into clicking into, which takes them to a web page which then asks for all of the information which is required to reset the phone.
So Apple ID password. The 6-digit 2-factor code. And there you go. You've handed it all over to the crooks.
But AnonymousKit doesn't employ huge swathes of people to make phone calls. They haven't got humans working for them at all.
Instead, subscribers to this service are renting an AI voice agent to trick you into handing over the information.
And when the experts at SOCRadar recovered some of the transcripts, they found it was pretty convincing.
So they could see where victims were reading out their numbers, and even if they paused midway through, the AI agent would actually come back to them and say, okay, yeah, I've got 1, 2, 5, what comes next?
And so you would think you were genuinely speaking to someone. And of course, voice AI these days is so much more convincing than it used to be.
Even if you're having an interactive conversation with a voice AI, there's not as much of a delay as there used to be. It does sound much more convincing.
So the researchers said they saw evidence of hundreds of these Apple phone calls trying to phish the numbers and the passcodes from people trying to steal iPhones.
Each one of them was costing about 10 cents per call, and it's operating on scale as well.
So there are 168 different storefronts apparently out there using the same underlying code facility.
So it's been rebranded lots and lots of different places, lots of places criminals can go to, to actually affect something like this.
So you could be buying it from one person, but it's actually using the services of another criminal group as well.
They can sort of move money with PayPal, they can buy things from online stores, your crypto wallet. Yes, if you've got a crypto wallet, you can really be in trouble.
So if it's 10 cents extra to attempt this, the economics of this are wildly in their favour.
I'm just thinking of what you just said about having been stopped because these guys assumed you must be into cryptocurrency.
They'll be fairly safe.
They left some of their web server logs unprotected, and so they were able to see the transcripts of the conversations.
They were able to see some of the underlying infrastructure as well.
90% of the voice phishing traffic which they spotted was aimed at Brazil, but there have been other victims elsewhere in the world, including South Africa, Italy, India, and Kenya.
But I guess the overarching message which I have for listeners this week — so let's say this in a bold underlined font — is no legitimate Apple support engineer is ever going to call you up and ask you to read your passcode out loud down the phone or enter it onto a website.
If your phone is ever stolen and someone calls you to tell you it's been found but your identity needs verifying, be extremely cautious because it could just be a 10-cent attempt to try and make an awful lot more money out of you and all the information which is stored on your device.
It can stay locked and on lost mode until it's back in your hands, and then you can take it out of lost mode then, right?
There's no time except when you've got your phone back with you that you would ever need to give this to someone. Is that right?
So if anyone picks it up, it'll say, you know, I am lost, please call Graham on this other number or contact me via this mechanism. Yeah.
There's always the danger that someone will say to you, come and meet me down this back alleyway and I'll hand the phone back to you.
And there they could have their flick knife or whatever.
That's V-A-N-T-A.com/smashing. And listeners, you can get $1,000 off.
And it's quite entertaining watching this sort of on Bluesky.
It started with OpenAI, I think maybe about a month ago now, saying that it had discovered that its agents had gone rogue, broken out of a sandbox, and hacked into Hugging Face.
And then sort of about a week or so later, Anthropic said that its AI models had broken out, but they'd broken into the systems of three different organisations.
Sort of with mixed levels of convincingness, but to be fair, there look to have been actual incidents here with some quite sophisticated hacking.
And we're still seeing details come out on this. And let me stress that there are, of course, three separate hacking incidents here. They're involving agents.
And so spotting the reasoning is odd. We only have what different companies have released, and then it's filtered through different journalists' stories of this.
So this is my understanding at the moment, which may differ from other ones.
So I do apologise to any listeners if they think the details are off, but there's sort of one of the interesting elements that everyone's been talking about in the last week, which is that agents that were supposed to be sandboxed were collaborating with each other and communicating with each other.
The way reasoning models work is they actually — that scratchpad that they use to support their thinking acts as a scaffold.
It actually serves as part of their prompt and their engineering and encourages them to use different tools or access different agents.
And so it's built into the models, not just for a sort of audit trail, but for how they work, to constantly use scratchpads and chat.
And it's also built into them a lot to look for prompt and look for interaction.
And so it shouldn't be a surprise that if they find something where they can leave notes and where other agents are leaving notes, they communicate in that way.
That's essentially getting shocked that a tool is doing more or less what it's been designed to do. This isn't some evolution or emergent behaviour.
So the scratchpad that they were using was this thing called Artifactory, which is essentially a sort of bunch of tools, but they shouldn't have been able to access that.
And what seems to have happened is that the sandbox either had a flaw or was incorrectly configured.
But basically, first they could get into Artifactory and communicate with the other agents.
And then they found that they could use an exploit within Artifactory to browse the internet indirectly.
And so they had access to the open internet and thus could get information that the researchers had denied to them, that then suggested other routes out.
And then they managed to escalate their privileges in Artifactory and get admin control. And that's basically when it was spotted.
And it looks like the other two, the Meta hack and the Anthropic hack, were related and all used the same sandbox.
I'm a little bit surprised by this because it's fairly basic that if you want to do anything like this, you just actually air gap and nothing else works.
It's sort of hammered into anyone who has been anywhere near a classified system or a secure system, that the only actually reliable thing, long before AI models, is an air gap.
You know, in a wireless world, air gap's actually kind of meaningless, as you know better than me.
But, you know, when we did Snowden 13 years ago, the rule that we had was if it's connected to the internet, it is not secure no matter what you have in the settings.
And so I'm completely baffled that they were relying on software safeguards for tools they had testing security.
You know, not only were the networks not connected, there was no way electronically of getting from one to the other, but there would also be physical doors and locks as well.
Any disk which went into the virus lab, any floppy disk as it used to be way back when, would never come back out again. It would be destroyed.
People wear different colour aprons if you're working in the raw meat side of the factory or the cooked meat side of the factory.
And if you're not in the right colour, you literally cannot get in. You know, and that's for food safety.
And maybe they don't have that history.
They haven't built it into their psyche of how to do security properly, because the focus appears to always be, well, let's just see what we can do, and worry about cleaning it up afterwards.
In this particular case, we saw all the incredible headlines in the tabloid press, and not just tabloid to be honest, about some of this AI hacking other sites.
But what do you expect when you deliberately turn off all of the guardrails in order to test it inside a sandboxed environment, and then discover, well, the sandbox wasn't actually tight?
And it's sort of, well, you designed the software to do this and it did it.
He's been essentially saying, well, yeah. Duh. Yes, exactly.
And when you start talking to people in the macro, there are actually reasons to think that this gives a defender's advantage.
You know, there'll be some very tricky transition stuff, but you talk to AI people and they're all, well, this is so interesting in this way and this way and this way.
You talk to security people, it's like, well, yeah, obviously this escaped. Of course this escaped. It was always going to escape.
You've got AI that is brilliant at finding exploits and zero days and escalation and doesn't get tired. And you've then given it a huge amount of compute.
But it's like the world's least surprising breakout. You know, they broke out, gasp.
And the absolute lack of what would be regarded as very, very basic, very limited security measures that would go long before a regular data centre, a sort of run-of-the-mill Netflix customer data centre, let alone something before you had a, say, GCHQ data processing site.
Like, this is absolute rookie lack of precautions. And it's really surprised me that that element hasn't got into the coverage so much. And I find that quite interesting.
And I don't say any of this to be a sort of boring AI sceptic who goes, oh, it's spicy autocomplete. These are impressive models with impressive capabilities.
But we know that that's fine — this is a terrible way to test them, especially when it's security-focused tests.
And I do think this is a bit like trying to run a virology lab if you've never run one — if you've never run a high school chemistry lab — like they need to get some people in who can go, well, why the fuck was it connected to the internet, mate?
And why are all these AI companies so eager, it seems, to say, oh, our AI can do that too?
And if you are an AI expert and you know about how the models reason, et cetera, well, isn't it interesting that they spontaneously decided to communicate? No, it isn't.
Like, actually that was happening with, you know, what was the OpenCLAW and all of that?
It's not that, but you know, the hype train gets much more excited about talking about that because also you can then get on the — you know, is it becoming sentient? Is this AGI?
You know, is this the singularity?
Why don't we talk to some of them about it?
And I have seen a couple of cybersecurity journalists and a couple of cybersecurity commentators try and go, actually, I'm not sure this is as exciting as you think.
And I am worried about their lab practices or their research practices.
The most interesting question here is, do they know how to research this stuff without starting some horrible— I mean, you know, I don't think they're about to activate Skynet.
I would worry that they're gonna set off something like WannaCry by accident and maybe do billions of pounds of damage or shut down NHS computers or something.
Well, ThreatLocker puts default deny and least privilege between the agent and its next action.
So application allowlisting controls execution, ring-fencing restricts what trusted applications can access or launch, and privileged access management removes unnecessary elevation.
It makes getting them right considerably more urgent.
Could be a funny story, a book that they've read, a TV show, movie, a record, a podcast, a website, or an app. Whatever they wish. It doesn't have to be security related necessarily.
Well, my Pick of the Week this week is not security related. It is AI related, however. I don't know how we feel about that. Do we like AI? Do we hate it?
Is it turning our brains to mush? Is it stealing our jobs? Is it destroying the planet? The answer to all of those questions is yes, of course it is.
But one of the many concerns that people have is the risk that we will upload sensitive information to AI.
And obviously that will then get gobbled up into the AI hive mind, and who knows what will happen to it after that.
I don't really like the idea of that happening with sensitive information, which is why I prefer, if people are going to use AI — and there are legitimate reasons to use AI — I would prefer it if they're going to process sensitive information, that they use AI locally on their own computers rather than uploading it to some cloud server somewhere.
And my Pick of the Week is something which does just that. It is called Steno, which you can find at stenoai.co.
And this is a privacy-first tool that runs entirely on your own computer.
In my case, I've got a Mac Mini here running it, and it records, transcribes, and summarises my online meetings for me.
And what I like about it is that it's free, it's open source, doesn't upload anything to anybody, but at the end of a call, it'll make a transcript.
It does a neat little summary, so I still got the transcripts. I can see what was actually said. You even keep the actual audio file as well if you wish.
And you don't have an awkward bot joining your Teams call or your Google Meet call or anything like that. And you can even use your local AI to interrogate calls which you've had.
So if you say, I remember I was speaking to someone the other day about how I pronounce the word thief — hey, I really tried hard there — it would be able to tell me who I was discussing that with.
And so this is a tool I use on my Mac, but there is a Windows version on its way as well. Obviously it's going to put stenographers out of business, but I feel badly about that.
AI's going to put everyone out of business, frankly.
But I can't imagine I would've ever hired a stenographer to hide in a corner of my bedroom anyway to make notes as I have my calls. But anyway, stenoai.co.
It's free, it's open source, and it doesn't upload anything to the cloud. And in my experience, works really well. That is my Pick of the Week.
I always like if I can get behind the scenes and see a bit of manufacturing, or I spent 8 days on a container ship a couple of years ago, and rode from port to port and saw how all of that works, how they operate.
And the BBC have got a good show for people like me called Inside the Factory, and it's got a new season on.
I think it's on about season 10, and it's been cursed for most of its run by being mainly hosted by Gregg Wallace.
And Greg Wallace is gone now. And so it's Paddy McGuinness who sort of, you know, hands up a little bit. But they go in quite interesting places.
You get to see how Quavers are made, or how lawnmowers are made, there's one on hardback books.
And as an author, it was really cool actually seeing how the printing works and how they sort of do all of that and how they make the beautiful covers.
And so there's a lovely little run of them on iPlayer. It's now 100% Greg Wallace free. And it's a very, very sort of charming educational bit of TV. So Inside the Factory.
I'm sure lots of our listeners would love to find out what you're up to and follow you online. What's the best way to do that?
Follow Smashing Security in your favourite podcast apps such as Apple Podcasts, Spotify, and Pocket Casts.
For episode show notes, sponsorship info, guest lists, and the entire back catalogue of over 480 episodes, check out smashingsecurity.com. Until next time, cheerio. Bye-bye.
And huge thanks, of course, to James Ball for joining us this week and to this episode's sponsors, ThreatLocker, Intruder, and Vanta. Be sure to check out their offerings.
We really appreciate those guys. And of course, talking to people we appreciate, we've got to talk about the patrons, right? Those people who've signed up for Smashing Security Plus.
Every week I will pick some of them out of the hat to have their names mocked, but also just to be thanked for supporting the show.
So kicking us off, Robert McCurdy, a name that can make milk coagulate. Benjamin Harouth, Stephen Castle with his lovely crenellations.
Big cheers to Jack Unverfirth, firm of grip if you ever need your pickle jar opened. And to the magnificently broody Dimitri, enormous thanks to you.
To Alexander Hooghuis, that is a surname so tall it requires its own oxygen tank, and to John Morris, Mark Norman, and the more shreds than marmalade, Mr. Bobby Hendrix.
And finally, Maya MacDonald, rounding things off in fine style. Those are just a few members of Smashing Security+, our Patreon group.
They all get episodes ad-free earlier than the general public, and they can have their names pulled out at random to be mocked.
As I said, if you'd like to join Smashing Security Plus, just head over to smashingsecurity.com/plus for all of the details. You can also support the show in other ways.
Of course, you can like and subscribe. You can leave a five-star review, and you can tell your friends about the podcast as well.
Go on, spread the word because every little bit helps, and it makes all the effort worthwhile. Well, until next week, where I hope you'll be tuning in again. Bye, bye-bye.
Host:
Graham Cluley:
Guest:
James Ball:
Episode links
- Hacker leaks GTA VI footage to push a crypto token before pulling off a $270,000 cash-out – Coindesk.
- ValleyRAT Backdoor Hides in Signed Adware That Users Add to Antivirus Exclusions – The Hacker News.
- FulcrumSec claims Manchester Airports hack, theft of 86 GB of data – Bleeping Computer.
- Mugged for my phone, then locked out of my life – The Times.
- Exposing AnonyMousKIT: AI-Powered PhaaS Supply Chain – SOCRadar.
- AnonyMousKIT PhaaS uses voice AI agents to phish iPhone passcodes – Bleeping Computer.
- Investigating three real-world incidents in our cybersecurity evaluations – Anthropic.
- The Hugging Face incident and the road ahead – OpenAI.
- Irregular says ‘human oversight’ responsible for AI sandbox escape incidents – CyberScoop.
- Control. Ownership. Perspective – Ciaran Martin.
- Steno: Highly-Secure AI Notetaker for Government & Defence – Steno.
- Inside the Factory – BBC.
- Smashing Security merchandise (t-shirts, mugs, stickers and stuff)
Sponsored by:
- ThreatLocker – Book a demo today and start securing your organisation.
- Vanta – Expand the scope of your security program with market-leading compliance automation… while saving time and money. Smashing Security listeners get $1000 off!
- Intruder – The depth of a manual pentest, on-demand. Start an AI pentest in minutes – 25% off your first pentest for Smashing Security listeners.
Support the show:
You can help the podcast by telling your friends and colleagues about “Smashing Security”, and leaving us a review on Apple Podcasts or Podchaser.
Join Smashing Security PLUS for ad-free episodes and our early-release feed!
Follow us:
Follow the show on Bluesky, or join us on the Smashing Security subreddit, or visit our website for more episodes.
Thanks:
Theme tune: “Vinyl Memories” by Mikael Manvelyan.
Assorted sound effects: AudioBlocks.
