The iPhone security setting that you should enable right now, the worrying way that AI is predicting what criminals look like, and we play a game of face fake or real…
All this and much more is discussed in the latest edition of the award-winning “Smashing Security” podcast by cybersecurity veterans Graham Cluley and Carole Theriault, joined this week by Mark Stockley.
Warning: This podcast may contain nuts, adult themes, and rude language.
0:00
0:00
0:00
0:00
Show full transcript ▼
This transcript was generated automatically, probably contains mistakes, and has not been manually verified.
It seems bonkers. I heard today about a blockchain—
Stop there!
A blockchain-assisted karaoke company.
Solving the problem—
What problem is there in karaoke that needs to be solved?
Smashing Security, episode 357. Interview with an iPhone thief, anti-AI, and have we Gone Too Far with Carole Theriault and Graham Cluley.
Hello, hello, and welcome to Smashing Security episode 357. My name's Graham Cluley.
Hello, hello, and welcome to Smashing Security episode 357. My name's Graham Cluley.
And I'm Carole Theriault.
And cluck, cluck, cluck, look who has joined us. He's come back for another week.
The chicken man.
Chicken man, Mark Stockley. Hello, Mark.
Oh, hi. What an introduction.
Cock-a-doodle-doo, what a treat it is to see you.
Yes. So for any listeners wondering what my credentials are for appearing on this podcast, it's because I own chickens. That's what you're saying.
I think you're more than an owner of chickens. You're an expert in all things hen-based. Penetration, perhaps. Oh no, penetration sounds a terrible thing. Okay.
I think we should kick this show off. But first, let's thank this week's wonderful sponsors, Collide and Vanta. It's their support that helps us give you this show for free.
Now, coming up in today's show, Graham, what do you got?
Now, coming up in today's show, Graham, what do you got?
I'm going to tell you how to steal an iPhone.
Mark, what about you?
I'll be asking, have we gone too far?
The answer is surely yes. And I will ask whether it is time for anti-AI. All this and much more coming up on this episode of Smashing Security.
Now, chums, chums, I don't know if either of you have ever been to London. London, big city in the center. It's not that far from us, but yeah, about an hour drive.
Yeah, by train. So yes.
We on the podcast, all of us have been to London. Some of our listeners in far-flung lands, maybe not.
But in London, take heed if you plan to take your smartphone, because someone in London has their iPhone stolen every 6 minutes.
But in London, take heed if you plan to take your smartphone, because someone in London has their iPhone stolen every 6 minutes.
How many iPhones does this person have?
He's getting really annoyed.
He's getting very, very annoyed.
Absolutely sick of it.
Now, it is bad enough that your iPhone can be physically stolen, but of course thieves can also do naughty things with your iPhone as well.
And that is the topic of my discussion today, is what thieves do to both steal your iPhone and what they can do afterwards, and maybe how you can better protect yourself into the future as well.
So there was a video which came out by the Wall Street Journal just before Christmas, where they interviewed an iPhone thief in his prison cell.
He'd been sent away for something, I don't know, 7 or 8 years or something for his part in an iPhone theft gang.
And that is the topic of my discussion today, is what thieves do to both steal your iPhone and what they can do afterwards, and maybe how you can better protect yourself into the future as well.
So there was a video which came out by the Wall Street Journal just before Christmas, where they interviewed an iPhone thief in his prison cell.
He'd been sent away for something, I don't know, 7 or 8 years or something for his part in an iPhone theft gang.
Aaron Johnson is due to spend the next several years at this high-security prison because he stole hundreds of iPhones and from them hundreds of thousands of dollars.
He exploited a vulnerability in Apple's software the same vulnerability I've been investigating for the last year.
He exploited a vulnerability in Apple's software the same vulnerability I've been investigating for the last year.
He grabbed the phone and then disappeared.
Along with others, Johnson, seen here in red, would target people in and around bars in Minneapolis to get their phones and their passcodes.
Was it a phone interview?
No, no, this was in person. He was in front of a camera. He did this.
Can I ask, does this have anything to do with the settings on your phone, or is this going to be the big reveal later?
We are going to be talking about iPhone settings later on. So, Carole, I would actually like you to reach for your iPhone. Mark, you can as well if you own an iPhone.
Yeah.
Turn it on. Don't accept any calls because I will be telling you later what settings you should change on your iPhone. So, ready yourself.
I'm ready. I'm ready.
All right.
Fantastic. So, this chap was interviewed and he explained what he did.
Well, what he did was he went out with a couple of his mates and they go out and hang out in bars where there were sort of young drunken people, you know, people who were partying, people having a good time, maybe slightly inebriated already.
Well, what he did was he went out with a couple of his mates and they go out and hang out in bars where there were sort of young drunken people, you know, people who were partying, people having a good time, maybe slightly inebriated already.
Did these guys fit into this environment or do they stand out like two sore thumbs?
No, no, no, they fitted in quite well.
It's not like he's 86, right?
No, no, no, no, no, no, no, no, no. He was of similar demographic.
Okay.
He's got a black mask with a couple of eye holes cut in it. It's got some sort of stripey jumper.
Oh, I see. I thought you were talking about gimps or some sort of sex club.
No, of course you did, Graham, of course you did. No, he hasn't.
Or anything like that. But there he is, right? He's just hanging out and he's pretending that he has drugs to sell.
It turns out he didn't actually have any drugs at all, but he's sort of pretending, right? Saying, oh yeah, I've got a bit of Shatner's bassoon.
It turns out he didn't actually have any drugs at all, but he's sort of pretending, right? Saying, oh yeah, I've got a bit of Shatner's bassoon.
Jazz cigarettes?
Exactly. Maybe something a little bit stronger as well. So he's suggesting that he has drugs to sell. And so he bumps into some young person who wants some drugs for the night.
Right.
And he says, look, I can get you some drugs and things.
He says, 'Why don't you take down my details?' And so the other person gets his phone out to write down his details, and he's got complicated details.
He says, 'Let me have your phone. Let me type it in for you,' right?
And of course, this inebriated person, this student who's having a great time, says, 'Sure,' maybe unlocks his phone, hands it over, right?
And after he's handed it over, the guy who's planning to steal it says, 'Oh, it's locked.' So he just quickly locks the phone. He says, 'Locked?
What's your passcode?' And people desperate for a bit of Colombian blacktail. Actually, I think that's a type of free-range egg, isn't it? Rather than a—
He says, 'Why don't you take down my details?' And so the other person gets his phone out to write down his details, and he's got complicated details.
He says, 'Let me have your phone. Let me type it in for you,' right?
And of course, this inebriated person, this student who's having a great time, says, 'Sure,' maybe unlocks his phone, hands it over, right?
And after he's handed it over, the guy who's planning to steal it says, 'Oh, it's locked.' So he just quickly locks the phone. He says, 'Locked?
What's your passcode?' And people desperate for a bit of Colombian blacktail. Actually, I think that's a type of free-range egg, isn't it? Rather than a—
It is, yeah.
That's for you, Mark. That's for you, Mark.
You know, when the munchies hit, the munchies hit. You need what you need.
Need some scrambled egg. Anyway, so before he poaches the phone, so he asks, he says, what's your passcode? He says, what's your passcode?
And people might just hand it over, and they may take back the phone and type it. But at that point, he watches them enter it.
Or, of course, people just say, "Oh, it's 2264813," or something like that, right? And so they enter their passcode. So, the villain now knows the passcode, and he knows what it is.
They're in a bar. The phone user's been drinking, already drunk. They're having a bit of a chitchat, and they're distracting him.
And at an opportune moment, the thief just passes the phone to one of his mates, and voomf, it's gone.
And people might just hand it over, and they may take back the phone and type it. But at that point, he watches them enter it.
Or, of course, people just say, "Oh, it's 2264813," or something like that, right? And so they enter their passcode. So, the villain now knows the passcode, and he knows what it is.
They're in a bar. The phone user's been drinking, already drunk. They're having a bit of a chitchat, and they're distracting him.
And at an opportune moment, the thief just passes the phone to one of his mates, and voomf, it's gone.
What?
And the guy doesn't go, "Can I have my phone back?" No, well, maybe he does, and he's like, "I haven't got your phone. I gave it back to you. I put it in your pocket," or whatever.
"I put it down here. Where's it gone?" You know, create a distraction. You make yourself scarce. It's a hubbling, bubbling kind of place.
"I put it down here. Where's it gone?" You know, create a distraction. You make yourself scarce. It's a hubbling, bubbling kind of place.
That's a good way to get a punch in the face, I think.
Well—
I don't know. Mark, what do you think?
Well, it turns out it's also a good way to get an iPhone.
Yes.
It depends what you're after.
Yeah, 'cause I suppose you don't have the iPhone anymore. You've handed it over to your mate. They've scarpered it. Exactly. And your phone's gone.
I haven't got it on me. Check me, you know, look in my pockets. I haven't got anything. Sorry, mate. I don't know what happened to it or whatever.
Anyway, immediately after the phone is stolen, what these guys do is they reset the passcode and they turn off Find My iPhone.
So that means that the genuine owner of the phone can't remotely track it or erase the device. And to do this, all you need is the passcode to do this.
The real owner no longer has any access to his phone. The next thing which the thieves do is they replace the real owner's face from Face ID and replace it with their own.
Anyway, immediately after the phone is stolen, what these guys do is they reset the passcode and they turn off Find My iPhone.
So that means that the genuine owner of the phone can't remotely track it or erase the device. And to do this, all you need is the passcode to do this.
The real owner no longer has any access to his phone. The next thing which the thieves do is they replace the real owner's face from Face ID and replace it with their own.
This is my phone, right? You've gotten to my settings. Where are you going from here?
So then I go add my face on there on the Face ID verification. Now when you got your face on there, you got the key to everything.
It's really opening things that people thought were safe. Like savings, check-ins, cryptocurrency apps, Venmo, PayPal. Yeah, you don't need face for none of that.
That's kind of little money. I'm trying to take as much as I can.
What this thief says is there are some things where maybe the face isn't enough to unlock it, but quite typically codes and passwords would be stored unprotected inside users' Notes app.
So they would just keep in plain text, in their Notes app, the one which regularly comes with the iPhone, a password, a passcode, something to unlock some account would often be there.
It's really opening things that people thought were safe. Like savings, check-ins, cryptocurrency apps, Venmo, PayPal. Yeah, you don't need face for none of that.
That's kind of little money. I'm trying to take as much as I can.
What this thief says is there are some things where maybe the face isn't enough to unlock it, but quite typically codes and passwords would be stored unprotected inside users' Notes app.
So they would just keep in plain text, in their Notes app, the one which regularly comes with the iPhone, a password, a passcode, something to unlock some account would often be there.
They would have a lot of fun searching my notes for any information that, because there's about 8 billion entries. So enjoy that one.
Really disorganised, is it?
Oh yeah.
I suppose you might have an entry which is Facebook colon and then your Facebook password. You know, some people might have that.
From 2005.
Well, you know. Maybe that's a bad example. But you could say search for a word Facebook.
The other thing which people do is sometimes they store those kind of passwords in their photos.
So they take a photograph of something, think, oh yeah, well, I'll put it in this folder and people won't look to look there.
But the thieves do look in those kind of places in order to find this stuff. But now, now they've unlocked your phone. Now they have control over your phone.
They can buy stuff, they can do stuff with Apple Pay, and they can ultimately, after they've caused their shenanigans, after they've logged into your bank account or done other things, which they're now able to do because they've set their biometrics up on your phone, they may ultimately wipe your iPhone, sell it to someone else, which makes them $900.
The other thing which people do is sometimes they store those kind of passwords in their photos.
So they take a photograph of something, think, oh yeah, well, I'll put it in this folder and people won't look to look there.
But the thieves do look in those kind of places in order to find this stuff. But now, now they've unlocked your phone. Now they have control over your phone.
They can buy stuff, they can do stuff with Apple Pay, and they can ultimately, after they've caused their shenanigans, after they've logged into your bank account or done other things, which they're now able to do because they've set their biometrics up on your phone, they may ultimately wipe your iPhone, sell it to someone else, which makes them $900.
Fun.
So the problem is the weak link is the passcode, right?
Is that you unlocked the phone and then with that information, which you've either given this guy who you innocently thought was going to sell you drugs or maybe some lovely free-range eggs, has instead actually scarpered off with it and now has access to your online accounts.
Well, about a week or so ago, a new version of the iPhone operating system came out called iOS 17.3, and it has a new security feature that many people might benefit from.
And this is why, Carole, I've told you to get your iPhone out, because Apple has not turned this on by default, and I'm recommending that everyone who has an iPhone turns this on because I think this is a good security feature.
It is called Stolen Device Protection, and what it does is it requires you to use Face ID, Touch ID, you know, some form of biometrics to unlock all the kinds of settings on your phone rather than your passcode.
So your passcode won't be enough. And this is specifically for when your phone is away from your workplace or your home.
So your iPhone has a way of learning, this is where he goes to work, this is where he is at home, by the regularity, I guess, of where you are.
Is that you unlocked the phone and then with that information, which you've either given this guy who you innocently thought was going to sell you drugs or maybe some lovely free-range eggs, has instead actually scarpered off with it and now has access to your online accounts.
Well, about a week or so ago, a new version of the iPhone operating system came out called iOS 17.3, and it has a new security feature that many people might benefit from.
And this is why, Carole, I've told you to get your iPhone out, because Apple has not turned this on by default, and I'm recommending that everyone who has an iPhone turns this on because I think this is a good security feature.
It is called Stolen Device Protection, and what it does is it requires you to use Face ID, Touch ID, you know, some form of biometrics to unlock all the kinds of settings on your phone rather than your passcode.
So your passcode won't be enough. And this is specifically for when your phone is away from your workplace or your home.
So your iPhone has a way of learning, this is where he goes to work, this is where he is at home, by the regularity, I guess, of where you are.
This is really irritating for me.
Why is that?
Because I don't use Face ID or Touch ID.
Oh, okay. You just use a code.
I use a code. I don't use a 4-digit code. It's longer than that. And it's a pain in the ass to get in and out of my phone. But then that's probably why I use it less than most people.
There's another setting for people like you. And that is simply, just don't say your passcode out loud to drug dealers.
Right.
Yeah, maybe don't buy drugs, girl.
Okay. Okay.
I'm in. Or if you do, don't take your iPhone. Don't go clubbing. Leave your iPhone at home.
Don't get drunk. Don't bring your iPhone.
Bring your 1990s Nokia with you instead.
Live a boring life. Okay. I will.
I am, in fact.
So, if you're not in a familiar location known to your iPhone, you'll be forced to wait for an hour before changing the passcode or turning off Find My iPhone or adding new biometrics like a new face to the phone, which gives obviously the person who's lost the phone the opportunity to set up the Lost Mode to remotely wipe the device and do all kinds of things if they want to.
So they just have to wait an hour.
Well, yeah, the criminal has to wait an hour, but that's critical because normally they find they have to do these things immediately.
Otherwise they won't be able to get back in.
They are locked out. So this biometric time delay means your passcode won't be enough to turn off Lost Mode. It won't be enough to erase the phone.
It won't be enough to access passwords or passkeys saved in Keychain. It won't let you look at payment methods stored in Safari Autofill.
The crooks won't be able to add their own face to Face ID. They won't be able to add their fingerprints as they don't have your existing biometrics.
So they would have to steal you and maybe chop off your finger as well or something like that in order to unlock your phone or take your eyeball, I suppose.
It won't be enough to access passwords or passkeys saved in Keychain. It won't let you look at payment methods stored in Safari Autofill.
The crooks won't be able to add their own face to Face ID. They won't be able to add their fingerprints as they don't have your existing biometrics.
So they would have to steal you and maybe chop off your finger as well or something like that in order to unlock your phone or take your eyeball, I suppose.
That doesn't work either, does it? Well, no, actually, I— Just for any drug dealer slash iPhone thieves listening.
Sharpening spoons out there.
Yeah.
Yeah. I think Apple Touch ID is meant to be able to tell that it's not a live finger, isn't it?
Yeah.
Yeah.
So maybe yeah, I don't know how long the finger has to be detached for in order for it. There's definitely something.
There's definitely something that's trying to work out if you're alive.
There's definitely something that's trying to work out if you're alive.
Something, isn't it? Yes, that's right.
So unlike now, you or your thief won't be able to fall back to the passcode entry to make those changes unless you're at your home or you're at your workplace.
And of course, hopefully, Carole, you are not losing your phone to thieves inside your home.
So unlike now, you or your thief won't be able to fall back to the passcode entry to make those changes unless you're at your home or you're at your workplace.
And of course, hopefully, Carole, you are not losing your phone to thieves inside your home.
I doubt anyone would want my phone. It's quite an old model, you know.
This podcast isn't purely for you, Carole.
You just keep bringing me up. I mean, you know.
Well, I'm just giving you an example. Okay, I still think you should turn this on. Actually, you haven't even got bloody Face ID and biometrics turned on, have you?
So this isn't— right, so for everybody else.
So this isn't— right, so for everybody else.
Every other human on the planet.
For every other human on the planet who actually uses their phone, you can update to iOS 17.3.
You should hopefully have already done that for now, and you can turn on Stolen Device Protection in Settings.
So you go into Settings, you tap Face ID and Passcode, you go into that submenu, you'll have to enter your device passcode, and then simply toggle Stolen Device Protection on.
You should hopefully have already done that for now, and you can turn on Stolen Device Protection in Settings.
So you go into Settings, you tap Face ID and Passcode, you go into that submenu, you'll have to enter your device passcode, and then simply toggle Stolen Device Protection on.
Yeah, and if it's not there, you maybe didn't update your phone yet, and naughty naughty chop chop.
Or you've still got an iPhone 4 and it doesn't support it or something like that.
That's right.
I think there's an even more interesting story in a story here.
Okay.
Because the setting is great. I think this is a great little step forward because you're already protected unless you're actually sharing your passcode or somebody's watching you.
Somebody gets your iPhone, it's basically locked.
You know, they've got 10 chances to unlock it, which they're not going to manage to do, and they can't get stuff out of it because it's all encrypted.
So your iPhone's already pretty safe. It's only for that fairly narrow situation where somebody managed to get your passcode as well.
What's really interesting about this to me is that there are now settings where the biometrics are the gatekeepers and not the passcode.
So for the whole time that we've been dealing with biometrics, it's always been the case that the biometric is backed up by a passcode or some sort of entry of a code of some kind.
A passcode is a yes/no answer. You either get the passcode right or you get it wrong.
But a biometric is a kind of "Okay, we think it's you, there's a high chance it's you." So it's a very different kind of assessment.
And we just didn't know 10 years ago, 12 years ago, we just didn't know how effective biometrics were going to be or how reliable they were going to be.
And Apple have now had biometrics on phones for at least a decade, starting with Touch ID. And they sell billions of these things.
So there are millions and millions and millions, hundreds of millions of phones out there that people have been using with biometrics for a decade.
So Apple must have an enormous amount of data about how effective biometrics are.
And they've taken this step now, which is the first time I can remember anybody doing it, of saying, actually, the passcode is backed up by the biometric rather than the biometric being backed up by the passcode.
And that, I think, is the thin end of a wedge that leads to passcodeless authentication.
Because it's saying we trust the biometric even more than we trust the passcode, which they would have good reason to determine that they can, then it seems to me that this could easily be a prototype for, all right, okay, well, we're going to let the biometric be the way that you access other sensitive things.
And ultimately, that leads to not having a passcode on the phone at all. And I think we spent enough time with biometrics now to know that actually they're pretty reliable.
And that could be where this goes.
Somebody gets your iPhone, it's basically locked.
You know, they've got 10 chances to unlock it, which they're not going to manage to do, and they can't get stuff out of it because it's all encrypted.
So your iPhone's already pretty safe. It's only for that fairly narrow situation where somebody managed to get your passcode as well.
What's really interesting about this to me is that there are now settings where the biometrics are the gatekeepers and not the passcode.
So for the whole time that we've been dealing with biometrics, it's always been the case that the biometric is backed up by a passcode or some sort of entry of a code of some kind.
A passcode is a yes/no answer. You either get the passcode right or you get it wrong.
But a biometric is a kind of "Okay, we think it's you, there's a high chance it's you." So it's a very different kind of assessment.
And we just didn't know 10 years ago, 12 years ago, we just didn't know how effective biometrics were going to be or how reliable they were going to be.
And Apple have now had biometrics on phones for at least a decade, starting with Touch ID. And they sell billions of these things.
So there are millions and millions and millions, hundreds of millions of phones out there that people have been using with biometrics for a decade.
So Apple must have an enormous amount of data about how effective biometrics are.
And they've taken this step now, which is the first time I can remember anybody doing it, of saying, actually, the passcode is backed up by the biometric rather than the biometric being backed up by the passcode.
And that, I think, is the thin end of a wedge that leads to passcodeless authentication.
Because it's saying we trust the biometric even more than we trust the passcode, which they would have good reason to determine that they can, then it seems to me that this could easily be a prototype for, all right, okay, well, we're going to let the biometric be the way that you access other sensitive things.
And ultimately, that leads to not having a passcode on the phone at all. And I think we spent enough time with biometrics now to know that actually they're pretty reliable.
And that could be where this goes.
Yeah.
Maybe, but it's interesting it's coming along at a time when we can trust images and visual representations of people less and less thanks to deepfakes and other AI tomfoolery.
But that goes back to the alive thing with the finger.
These things, there's so much more going on than it just looking at — I can't just hold up a photo of Graham and log into his phone with Face ID.
It's actually trying to work out that you're a real person. And the same with the finger.
These things, there's so much more going on than it just looking at — I can't just hold up a photo of Graham and log into his phone with Face ID.
It's actually trying to work out that you're a real person. And the same with the finger.
Yes, you have to stick the picture of me on the front of a balloon, I think, to make it appear a little bit —
Yeah, or wear it as a mask.
Yes, have it bobbling around a little bit to make it more convincing.
Hi, hi everybody.
Well, we'll have to wait and see if Android adopts a similar feature. I suspect they might in the coming months, so we'll have to see that.
But certainly for our iPhone-loving listeners out there, I think this may be a sensible setting for them to turn on, particularly that guy in London who keeps losing his phone every 6 minutes.
Mark, what have you got for us this week?
But certainly for our iPhone-loving listeners out there, I think this may be a sensible setting for them to turn on, particularly that guy in London who keeps losing his phone every 6 minutes.
Mark, what have you got for us this week?
I've got a question.
Mm-hmm.
So I would like to ask you today, have we gone too far?
Frequently.
The answer is yes.
Mostly on your episodes.
I don't know about you, but I find myself increasingly looking at technology and thinking, have we gone too far? Is this actually useful? Is this even harmful?
And perhaps more often than both of those, are we actually allowed to be in charge of this stuff? Who left us in charge?
I mean, when you think about modern technology, it's the sum of thousands and thousands of years of this accumulated science and engineering, all this human endeavor, and it's each generation learning from the previous generation.
Nobody's just inventing things from scratch. For most of our history, flint knapping was the absolute pinnacle of technology, you know, for millions of years.
And then it took a very, very long time for us to get to things like pencils and chairs.
So we had all this time to kind of get used to this newfangled technology and figure out how to use it and what was safe and all this kind of thing.
And perhaps more often than both of those, are we actually allowed to be in charge of this stuff? Who left us in charge?
I mean, when you think about modern technology, it's the sum of thousands and thousands of years of this accumulated science and engineering, all this human endeavor, and it's each generation learning from the previous generation.
Nobody's just inventing things from scratch. For most of our history, flint knapping was the absolute pinnacle of technology, you know, for millions of years.
And then it took a very, very long time for us to get to things like pencils and chairs.
So we had all this time to kind of get used to this newfangled technology and figure out how to use it and what was safe and all this kind of thing.
I still haven't completely worked out how to use a chair, but I am alarmed that it appears Elon Musk is in charge and is now implanting things in people's brains.
That seems a concern.
That seems a concern.
I thought you were trying to say that Elon Musk was in charge of your chairs and that's why you couldn't sit on them.
Give him time. Give him time.
You can't pin that one on him, okay?
Not yet.
He's absolutely responsible for Twitter, but your inability to sit in a chair, that's between you and your parents, I'm afraid. Anyway, what am I trying to say?
Yeah, so it used to be that we have plenty of time to keep up with these innovations.
But now these days, I'm often left with this kind of nagging sense that modern technology has far exceeded our individual competence.
I have a fairly rudimentary car, and every time I sit in my car, I think, I know that my level of driving is not up to the level of engineering of my car.
You know, I barely kind of scratch the surface of what my laptop can do. And I just kind of feel we're not designed for Facebook and nuclear weapons and things like that.
I just feel like we're kids that have been left at home by our parents with 200 cigarettes and some heavy machinery or something like that.
So anyway, you can see where I'm going with all of this. So today I want to play a game called, have we gone too far?
And I want to start by introducing you to a company called Parabon NanoLabs.
Yeah, so it used to be that we have plenty of time to keep up with these innovations.
But now these days, I'm often left with this kind of nagging sense that modern technology has far exceeded our individual competence.
I have a fairly rudimentary car, and every time I sit in my car, I think, I know that my level of driving is not up to the level of engineering of my car.
You know, I barely kind of scratch the surface of what my laptop can do. And I just kind of feel we're not designed for Facebook and nuclear weapons and things like that.
I just feel like we're kids that have been left at home by our parents with 200 cigarettes and some heavy machinery or something like that.
So anyway, you can see where I'm going with all of this. So today I want to play a game called, have we gone too far?
And I want to start by introducing you to a company called Parabon NanoLabs.
Parabon NanoLabs.
Which describes itself as a vertically integrated DNA technology company.
Vertically integrated.
Yeah.
Okay, this feels like corporate bingo.
Do you want me to spell it?
No, I'm just trying to make sense of the fucking words coming out of your mouth.
It seems bonkers. I heard today about a blockchain—
Stop there, stop there.
A blockchain-assisted karaoke company.
Solving the problem. What problem is there in karaoke that needs to be solved? No, for real.
There is. It is a Korean crypto karaoke platform called Sumsing. And they've just been hacked.
No, I think this is a great example. Have we gone too far? What is this? Who asked for this?
I fear there's a midlife crisis happening right live on the show.
Yeah, you're witnessing it. Oh, it's the old Douglas Adams thing, isn't it?
You know, anything that exists when you're born is oxygen and everything else is just terrifying and needs to be burned. So yeah, so Parabon—
You know, anything that exists when you're born is oxygen and everything else is just terrifying and needs to be burned. So yeah, so Parabon—
Yes.
Parabon NanoLabs, they're a vertically integrated DNA technology company and it does lots of things.
And one of the things it does is it helps the police by linking DNA and genealogical data to help solve cold cases.
And one of the things it does is it helps the police by linking DNA and genealogical data to help solve cold cases.
Oh, okay.
Sounds like a good idea.
I'm sure everybody who has a family member, however distant, who's done some crap, loves being part of this. Okay, yeah.
Anyway, so I'll give you an example. This is an easy one to start with.
So, in 2019, an 82-year-old handyman was arrested for a rape and a double murder that he had committed 43 years earlier, thanks to Parabon NanoLabs.
So, the police were able to track him down after the labs uploaded some DNA from the crime scene to a public genealogy website called GEDmatch, which does genealogy and family trees.
And this established a family link to the Green Bay area in Wisconsin. And police zeroed in on the area, and they got a DNA sample. This is quite fun, this.
They got a DNA sample from the suspect by asking him to fill out a policing survey. And then he had to put the policing survey in an envelope, and lick on the—
So, in 2019, an 82-year-old handyman was arrested for a rape and a double murder that he had committed 43 years earlier, thanks to Parabon NanoLabs.
So, the police were able to track him down after the labs uploaded some DNA from the crime scene to a public genealogy website called GEDmatch, which does genealogy and family trees.
And this established a family link to the Green Bay area in Wisconsin. And police zeroed in on the area, and they got a DNA sample. This is quite fun, this.
They got a DNA sample from the suspect by asking him to fill out a policing survey. And then he had to put the policing survey in an envelope, and lick on the—
Lick the envelope.
It was nothing to do with the survey.
I always use a wet sponge.
Or your pet dog. You could use that if you really want to mess up the DNA test.
So the police cracked a cold case using Parabon NanoLabs and an envelope.
Hmm.
Seems like a good thing, right? Good thing.
Seems like it. Good thing.
Yeah. Yeah, I reckon. Yeah, here's another one. Okay. So there are other things that you can do with DNA to catch crooks. And Parabon NanoLabs will do those too.
And one of the things it can do is it will produce what it calls a snapshot phenotype report, which tells you what you can learn about someone's appearance from their DNA.
And one of the things it can do is it will produce what it calls a snapshot phenotype report, which tells you what you can learn about someone's appearance from their DNA.
Ooh.
And in fact, they did this for police back in 2017 in trying to identify the killer of a woman who'd been murdered 30 years earlier.
Wow.
So the police sent the lab DNA from a crime scene, and the company's AI algorithm— because you know we weren't going to get through this story without mentioning an AI algorithm— predicted that the murderer was male, had fair skin with no freckles, brown eyes, brown hair, and bushy eyebrows.
And just to clarify, Parabon NanoLabs was helping them. It didn't help them solve the case. It was just helping them—
And just to clarify, Parabon NanoLabs was helping them. It didn't help them solve the case. It was just helping them—
Pointed them in the right direction.
Get that far, yeah, yeah.
But with 30 years under the guy's belt, surely he's maybe gone a bit grey.
Oh, well done, Carole. Very, very, very good point. Good point.
That's a good point.
That's a good point.
So— Maybe his eyebrows will have molted a little as well.
Is that what's happening, Graham?
Mine are just getting longer.
But, you know, so now we can say, you know, thanks to the DNA, we can say at least the murderer was male.
They've got fair skin, they don't have freckles, they've got brown eyes, they've got brown hair, they've got bushy eyebrows. That sounds pretty useful, right?
They've got fair skin, they don't have freckles, they've got brown eyes, they've got brown hair, they've got bushy eyebrows. That sounds pretty useful, right?
Right. I'm beginning to feel a little bit worried. Maybe it's your mention of the words, you know, artificial intelligence in there where I just began to think, well—
Well, it's interesting that you just mentioned that, because I had the same reaction as you.
So the good thing is now the police have an idea about what this person probably looks like.
Or at least they've got an idea what an AI thinks this person looks like, and that might be useful, right?
So the good thing is now the police have an idea about what this person probably looks like.
Or at least they've got an idea what an AI thinks this person looks like, and that might be useful, right?
Might be.
Descriptions of crooks based on DNA. Good thing, bad thing?
As long as hallucinations don't get in there.
If they'd done a test on billions of people's DNA and found it to be 99% reliable or something — or let's hope actually a much larger percentage than 99% reliable — then that would be a good thing.
You know, if it was beyond reasonable doubt and say, "Yes, we are absolutely certain this isn't someone who's blue-eyed, but someone who's brown-eyed," then that would be helpful, I suppose.
You know, if it was beyond reasonable doubt and say, "Yes, we are absolutely certain this isn't someone who's blue-eyed, but someone who's brown-eyed," then that would be helpful, I suppose.
Yeah, you could imagine how that might help police with their inquiries, as the saying goes. Now, I left out a detail about the report that the lab produces.
So, the AI doesn't just produce a description of the person. It also creates a 3D render of their face, so you can actually see what the AI is guessing the person looks like.
So, the AI doesn't just produce a description of the person. It also creates a 3D render of their face, so you can actually see what the AI is guessing the person looks like.
Tell me you have some examples.
So the lab produced this 3D render of what they thought the murderer looked like.
And then they added some bits too, because DNA can't tell you about things like hairstyles and things like that, so the lab had a forensic artist add in a haircut and a moustache.
And then they added some bits too, because DNA can't tell you about things like hairstyles and things like that, so the lab had a forensic artist add in a haircut and a moustache.
Right. Of course, a moustache.
They got them from witness statements, so it's not without reason, right.
And the company, to your earlier point, the company produced two versions — they produced one of the guy aged 25, and another one of the guy aged 55.
And the police actually published those faces in an attempt to jog the public's memory.
And I should mention as well, to be fair, that the police are well aware that these might not be accurate.
So in the — I read a local news report from the local paper, and they had actually told the local papers at the East Bay Times that the composites were scientific approximations and not likely to be exact replicas.
And of course, environmental factors like smoking and drinking and diet and other things can't be predicted by DNA.
Although I think even that's an interesting question, because I can imagine that in future we might try to make predictions like that.
You know, based on your DNA, you have a propensity to addiction, and therefore you probably eat too many pizzas. Yeah, maybe you're overweight.
And I can see a future where we do actually try and make those predictions — I think that's where this slippery slope leads. So where are you at now?
What do you think now — good thing, bad thing? How are you feeling about this technology?
And the company, to your earlier point, the company produced two versions — they produced one of the guy aged 25, and another one of the guy aged 55.
And the police actually published those faces in an attempt to jog the public's memory.
And I should mention as well, to be fair, that the police are well aware that these might not be accurate.
So in the — I read a local news report from the local paper, and they had actually told the local papers at the East Bay Times that the composites were scientific approximations and not likely to be exact replicas.
And of course, environmental factors like smoking and drinking and diet and other things can't be predicted by DNA.
Although I think even that's an interesting question, because I can imagine that in future we might try to make predictions like that.
You know, based on your DNA, you have a propensity to addiction, and therefore you probably eat too many pizzas. Yeah, maybe you're overweight.
And I can see a future where we do actually try and make those predictions — I think that's where this slippery slope leads. So where are you at now?
What do you think now — good thing, bad thing? How are you feeling about this technology?
Feeling a little bit queasy, a little bit nervous.
Why? How are you feeling about this, Carole?
Oh no, no, I'm totally comfortable with all new tech. Everyone knows that. Yeah, yeah, yeah. I don't care at all.
I'm not being set up. Now, should I mention, it may help your queasiness, Graham, if I just mentioned that this technology obviously hasn't been peer-reviewed?
So, I mean, it sounds science-based, right? Paraben Labs have tested this.
So, I mean, it sounds science-based, right? Paraben Labs have tested this.
Well, they have labs. The fact they have labs in their name, that makes me think they're proper scientists — they're experts, yes.
I mean, just to be clear, we're not saying they're not experts. We're not saying they aren't scientists or they don't have a lab.
And they have tested this, and obviously they think that it works. And they're happy enough to sell this to the police. But it hasn't been peer-reviewed.
So maybe it's really accurate. And maybe it isn't.
And they have tested this, and obviously they think that it works. And they're happy enough to sell this to the police. But it hasn't been peer-reviewed.
So maybe it's really accurate. And maybe it isn't.
Yeah, a lot of stuff is not peer-reviewed. It's like you're taking their word for it, I guess. Or the cop's word as well. Users, the customers' testimonials, right?
Yeah. Our podcast is peer-reviewed.
Have you done a double-blind trial of this podcast? How do you do that?
We've had lots of reviews on Apple Podcasts, good and bad. And you know, generally people like us, of those people who choose to listen and leave a review.
Well, they like one of us.
That's definitely how you do a double-blind. Yeah.
A lot of them prefer Carole to me. But other than that.
So anyway, the technology hasn't been peer-reviewed. And maybe it's like face recognition. Maybe it is really good at white people. You know, you don't know.
Like sometimes these things have, depending on the training data, there can be kind of blind spots and weak spots and things like that. We just don't know.
And we shouldn't forget, again, to your earlier point, Graham, that this was all done by a machine learning model.
And machine learning models suffer from what we call the black box problem, which means that we don't actually know how they make decisions.
We know that if you feed in a certain type of input, you'll get a certain type of output, but we don't actually know what's going on under the hood to the point where we can say, okay, well, it decided that this person looks like this based on the DNA because it made these specific decisions.
So it's already looking pretty obscure, which doesn't mean it doesn't work, but it is a very opaque process.
Now, that said, I think it's important to remember that we trust witnesses and sketch artists.
And they come up with pictures of what suspects look like, and there's no science at work there at all.
Like sometimes these things have, depending on the training data, there can be kind of blind spots and weak spots and things like that. We just don't know.
And we shouldn't forget, again, to your earlier point, Graham, that this was all done by a machine learning model.
And machine learning models suffer from what we call the black box problem, which means that we don't actually know how they make decisions.
We know that if you feed in a certain type of input, you'll get a certain type of output, but we don't actually know what's going on under the hood to the point where we can say, okay, well, it decided that this person looks like this based on the DNA because it made these specific decisions.
So it's already looking pretty obscure, which doesn't mean it doesn't work, but it is a very opaque process.
Now, that said, I think it's important to remember that we trust witnesses and sketch artists.
And they come up with pictures of what suspects look like, and there's no science at work there at all.
No. It's like nose is a bit bigger. A little bit bigger. Bigger, smaller. Yeah.
So while I share your queasiness, Graham, I'm also sat here thinking, is this method actually any worse than just, you know— But it's not like—
It's not gonna be used on its own, this tool. It will be used presumably with other tried and tested methodologies in order to, you know, find someone guilty of a crime.
No? Yeah, we'd all like to believe that's how police work goes on. Yeah, no, I absolutely— I choose to believe that you're right.
All we can say is that in this instance, they've created these two renderings and they published them in the newspaper in an attempt to get people to come forward.
So presumably at that point then they would be using other evidence, perhaps like the DNA evidence that we were talking about earlier, which has got a little bit more rigor behind it.
But actually this story doesn't quite finish there because evidently the police didn't get any consequential leads from the renderings.
And in 2020, so 3 years later, a detective contacted the Northern California Regional Intelligence Center, which is a place that facilitates collaboration between different law enforcements, and I guess has access to some technology.
And they said, I've got a photo of a possible suspect, meaning this rendering, and we'd like to use facial recognition technology to identify a suspect or lead.
Now, so we don't know what happened next, because we only know about this, because it was part of a big data leak, and this happened to be one of the pieces of information.
This request happened to be one of the pieces of information. So we don't know what happened after this. We don't know if they ran the facial recognition, but we know they wanted to.
We don't know if it led to anything, but we also don't know how commonplace this is. Yeah, this is insane.
All we can say is that in this instance, they've created these two renderings and they published them in the newspaper in an attempt to get people to come forward.
So presumably at that point then they would be using other evidence, perhaps like the DNA evidence that we were talking about earlier, which has got a little bit more rigor behind it.
But actually this story doesn't quite finish there because evidently the police didn't get any consequential leads from the renderings.
And in 2020, so 3 years later, a detective contacted the Northern California Regional Intelligence Center, which is a place that facilitates collaboration between different law enforcements, and I guess has access to some technology.
And they said, I've got a photo of a possible suspect, meaning this rendering, and we'd like to use facial recognition technology to identify a suspect or lead.
Now, so we don't know what happened next, because we only know about this, because it was part of a big data leak, and this happened to be one of the pieces of information.
This request happened to be one of the pieces of information. So we don't know what happened after this. We don't know if they ran the facial recognition, but we know they wanted to.
We don't know if it led to anything, but we also don't know how commonplace this is. Yeah, this is insane.
I mean, it's madness enough if it only happened once, but the thought that it could have happened more than once.
Of people taking an image generated by a computer based on someone's spit, and then someone else says, oh, well, I'll run that through the facial recognition database and see who we come up with.
Yeah. I don't know.
Of people taking an image generated by a computer based on someone's spit, and then someone else says, oh, well, I'll run that through the facial recognition database and see who we come up with.
Yeah. I don't know.
I think it's—
You like it. You're okay. You like this.
You're comfortable with this. No, I don't like it.
I don't like it. But I can see there's also a problem of innocent people going to jail a lot. You know, that sucks.
And if that, you know, so there's pros and cons to this method, I'm sure. But it is scary. Of course it's scary.
And if that, you know, so there's pros and cons to this method, I'm sure. But it is scary. Of course it's scary.
So this makes me really queasy, really queasy, because basically what we're talking about is how do you feel about an AI, which is a black box, generating a guess about what somebody looks like?
Remember, it's not just saying, oh, they've got brown hair. It's saying they look like this. Here is a picture of their face.
And then another AI takes that and makes a guess about who that resembles.
I mean, this at a time when we have governments all around the world, or sort of local municipalities and local governments and things, banning the use of facial recognition, which by itself has proved to be really problematic because of things like biases in the training data.
And there's an old saying, Graham will know this because he's even older than I am, but there's an old saying in computing, garbage in, garbage out.
And the concern is, I don't think— so it's actually against Parabon NanoLabs' terms of service to do this.
So just the police in the police report were fully aware that the rendering that was produced was not— it's not likely to be an exact replica of the person, it's there to jog someone's memory.
Parabon NanoLabs understand the limits of what they're producing, and so it's not supposed to be used for things like facial recognition.
The problem seems to be, according to Wired, certainly, which is where I discovered the story, that there are no federal rules that limit the types of images that police can use with face recognition software.
So it can use fake AI pictures. Seemingly, yes. And it's not just fake AI pictures.
So law enforcement agencies have used blurry surveillance camera shots, manipulated photos of suspects. The sketches made by artists have been run through photo recognition.
And my favorite one, they've even used a picture of Woody Harrelson because in one case, the suspect looked like Woody Harrelson. So they ran that through.
So, and this is what I was talking about at the beginning. You know, are we really, you know, are we allowed to use this stuff?
Remember, it's not just saying, oh, they've got brown hair. It's saying they look like this. Here is a picture of their face.
And then another AI takes that and makes a guess about who that resembles.
I mean, this at a time when we have governments all around the world, or sort of local municipalities and local governments and things, banning the use of facial recognition, which by itself has proved to be really problematic because of things like biases in the training data.
And there's an old saying, Graham will know this because he's even older than I am, but there's an old saying in computing, garbage in, garbage out.
And the concern is, I don't think— so it's actually against Parabon NanoLabs' terms of service to do this.
So just the police in the police report were fully aware that the rendering that was produced was not— it's not likely to be an exact replica of the person, it's there to jog someone's memory.
Parabon NanoLabs understand the limits of what they're producing, and so it's not supposed to be used for things like facial recognition.
The problem seems to be, according to Wired, certainly, which is where I discovered the story, that there are no federal rules that limit the types of images that police can use with face recognition software.
So it can use fake AI pictures. Seemingly, yes. And it's not just fake AI pictures.
So law enforcement agencies have used blurry surveillance camera shots, manipulated photos of suspects. The sketches made by artists have been run through photo recognition.
And my favorite one, they've even used a picture of Woody Harrelson because in one case, the suspect looked like Woody Harrelson. So they ran that through.
So, and this is what I was talking about at the beginning. You know, are we really, you know, are we allowed to use this stuff?
Suddenly, a blockchain-based social karaoke platform doesn't seem such a bad idea. Crow, what have you got for us this week?
Oh, I'm going to lighten this up a little bit, I think. I've got some images here from the New York Times of people, of people's faces. And some of them are real.
Some of them may be AI-generated. It's the AI show today. And you guys are going to tell me what you think. Okay, so fake or real?
Is this a real person or is this a fake headshot of somebody, someone completely made up? And you guys are pretty bright, right? You're pretty bright in all stuff digital.
So let's put your expert eyes to the test and see if you can identify if someone is real or fake. So we've got number 1.
Some of them may be AI-generated. It's the AI show today. And you guys are going to tell me what you think. Okay, so fake or real?
Is this a real person or is this a fake headshot of somebody, someone completely made up? And you guys are pretty bright, right? You're pretty bright in all stuff digital.
So let's put your expert eyes to the test and see if you can identify if someone is real or fake. So we've got number 1.
Play along if you at home while you're listening.
Well, you can because the link's in the show notes and you can go do that.
Oh, okay, okay, great. Yes, you can. So you've got number 1.
You've got a young guy here. And basically, you know, is he a real guy or is he a fake guy? He's got a light beard.
I'm going to say fake.
You're going to say fake? Fake. Okay. What about you, Mark?
I'm saying AI.
Okay. You're both wrong. He's real. Next. Ooh. Was this made by AI? Fake or real? Another guy. Another chap.
Fake.
Fake?
Same.
I'm saying real.
OK, it's fake. It's fake. 92% of people got that one wrong. The first one, 86% of people got that wrong. OK, next one. OK, AI or fake?
She looks a bit scary.
She does look a little bit scary.
I'm going to say fake. I think she looks like she was generated by AI, and I'm now getting the feel for where this is going. So I'm going to say that she's real.
She's fake. And 84% of people got it wrong. Okay, next one. We've got two more. Next one. We've got a nice lady here with some glasses on.
Fake. I'm saying fake. Definitely real.
She's definitely real. She's definitely real. Okay, Mark. She's fake. 93% of people got it wrong. And finally, the last one. And a middle-aged guy.
Old fella. Looks like he's been out for a run. He's regretting it.
He's looking a little stressed.
Yeah, he's looking exhausted.
He's definitely fake. Fake. He's real. Fake.
Oh no. Okay, so— Well done, Mark. So participants were also asked to indicate how sure they were in their selections, right? So are you really sure?
And researchers found that the higher confidence correlated with a higher chance of being wrong. Oh. In other words, misguided with confidence.
And researchers found that the higher confidence correlated with a higher chance of being wrong. Oh. In other words, misguided with confidence.
You're not the first person to say that to me.
Now, apparently, AI research published across multiple studies found that faces of white people— one of you mentioned this earlier, I think it was Mark in your story— faces of white people created by AI systems are typically perceived as more realistic than genuine photographs of white people.
It's a phenomenon called hyperrealism. Okay, this is according to The New York Times. And this hyperrealistic face idea, these faces tend to be less distinctive, researchers say.
And so they closely average out the proportions.
And because of that, they fail to arouse suspicions amongst participants because we seem to fixate on features that drift away from average proportions.
So if someone has a big hook nose, you'd be gotta be fake. Gotta be fake. Or a misshapen ear. Interesting, huh? Hmm. So takeaway one, don't believe anything you can see online again.
But as we know, it's not just imagery, it's also AI-generated text.
So just last September, a study led by two experts in applied linguistics conducted some research to see if their counterparts could tell the difference between a research abstract written by a student, a human, or a machine.
And this is not a bad idea, because if anybody is going to be able to identify human-produced writing, it should be an expert in linguistics, right? That's what they do.
They spend their careers studying patterns in language and other aspects of human communication. You'd think so.
It's a phenomenon called hyperrealism. Okay, this is according to The New York Times. And this hyperrealistic face idea, these faces tend to be less distinctive, researchers say.
And so they closely average out the proportions.
And because of that, they fail to arouse suspicions amongst participants because we seem to fixate on features that drift away from average proportions.
So if someone has a big hook nose, you'd be gotta be fake. Gotta be fake. Or a misshapen ear. Interesting, huh? Hmm. So takeaway one, don't believe anything you can see online again.
But as we know, it's not just imagery, it's also AI-generated text.
So just last September, a study led by two experts in applied linguistics conducted some research to see if their counterparts could tell the difference between a research abstract written by a student, a human, or a machine.
And this is not a bad idea, because if anybody is going to be able to identify human-produced writing, it should be an expert in linguistics, right? That's what they do.
They spend their careers studying patterns in language and other aspects of human communication. You'd think so.
Right.
Well, each expert was asked to examine 4 writing samples and had to identify whether it was machine-made or human-made. None correctly identified all 4, and 13% got them all wrong.
And based on the larger findings— links in the show notes— the researchers concluded the professors would not be able to distinguish between a student's own writing and the writing generated by an AI-powered language such as ChatGPT without the help of software that hasn't yet been developed.
And maybe that's the key, right? Some authentication or defensive tools, some anti-AI.
And based on the larger findings— links in the show notes— the researchers concluded the professors would not be able to distinguish between a student's own writing and the writing generated by an AI-powered language such as ChatGPT without the help of software that hasn't yet been developed.
And maybe that's the key, right? Some authentication or defensive tools, some anti-AI.
I'm sure somebody released something last year.
Yeah, but it's not, I don't know of it being used. Are colleges being able to use this stuff? And probably they are.
There's probably some tools out there, but without the tools, basically research is saying we have no hope in hell.
And our little experiment here with the two of you showed that as well.
There's probably some tools out there, but without the tools, basically research is saying we have no hope in hell.
And our little experiment here with the two of you showed that as well.
I don't know how far it went, but the research I saw last year, it was about identifying patterns in language produced by AI.
And because there are patterns, just like anything else, there are patterns in the way that it— like you say with the photographs, it's producing an average of all the text that it's read.
Composite. Yes, it's a composite of everything that's read. And so there are things within that that another AI can spot.
But then you are in the hands of, well, let's hope our AI works better than their AI.
And because there are patterns, just like anything else, there are patterns in the way that it— like you say with the photographs, it's producing an average of all the text that it's read.
Composite. Yes, it's a composite of everything that's read. And so there are things within that that another AI can spot.
But then you are in the hands of, well, let's hope our AI works better than their AI.
Counting the AI-ness.
Just this past week, there was a report from GCHQ's National Cybersecurity Centre's chief, Lyndsay Cameron, and she's warning that AI is going to make the digital landscape much harder to protect.
I think all three of us would agree with that.
She says, quote, "The emergence, use of AI in cyberattacks is evolutionary, not revolutionary, meaning that it enhances existing threats like ransomware but does not transform the risk landscape in the near term." What do you think about that?
Just this past week, there was a report from GCHQ's National Cybersecurity Centre's chief, Lyndsay Cameron, and she's warning that AI is going to make the digital landscape much harder to protect.
I think all three of us would agree with that.
She says, quote, "The emergence, use of AI in cyberattacks is evolutionary, not revolutionary, meaning that it enhances existing threats like ransomware but does not transform the risk landscape in the near term." What do you think about that?
I wonder what she means by near term. How long is that?
Well, it's really complicated because the thing is, while ChatGPT's rules prohibit its use for spam or generating malicious code and all this stuff, researchers have found a way to bypass these controls.
I found a way to bypass these controls.
I found a way to bypass these controls.
It's not hard, yeah.
I wrote an article about 3 months ago. I got ChatGPT-3 to write some ransomware for me.
Did you?
And it was absolutely terrible. Garbage.
You would have to be a good enough programmer to write ransomware in order to put together the kind of fragments of nonsense that it spat out.
And then I repeated it 6 months later with ChatGPT-4. And let me tell you, ChatGPT-4 is really good at writing ransomware.
And there were some controls in place that were designed to stop you.
So you can't just rock up and say, write me some ransomware, because it'll go, hang on, I'm not allowed to do that.
But what I was able to do is just say, okay, well, what does ransomware do? Okay, it does X, Y, Z. So I said, right, write me a computer program that does X. And it said, fine.
And then I said, right, add Y. And then it, okay, fine, add Z. And I just added all of the common features that you find in ransomware. And then it made it.
And then I executed it on a virtual machine to make sure it worked. And it did.
You would have to be a good enough programmer to write ransomware in order to put together the kind of fragments of nonsense that it spat out.
And then I repeated it 6 months later with ChatGPT-4. And let me tell you, ChatGPT-4 is really good at writing ransomware.
And there were some controls in place that were designed to stop you.
So you can't just rock up and say, write me some ransomware, because it'll go, hang on, I'm not allowed to do that.
But what I was able to do is just say, okay, well, what does ransomware do? Okay, it does X, Y, Z. So I said, right, write me a computer program that does X. And it said, fine.
And then I said, right, add Y. And then it, okay, fine, add Z. And I just added all of the common features that you find in ransomware. And then it made it.
And then I executed it on a virtual machine to make sure it worked. And it did.
Jesus.
So yeah, it will absolutely help you write malware. And the danger there is not that it's— things like ransomware is feature complete.
So if you look at the actual ransomware executables, they haven't changed very much in several years because they do everything that the crooks need them to do.
So it's very unlikely that AI is going to come along.
And this is maybe we haven't seen an explosion of AI in cybersecurity, certainly not amongst cybercriminals, because at the moment they just don't seem to need it.
So ransomware does what it needs to do. So you're not going to get an AI come along and write a better ransomware because it's not going to get much better than it is.
But what it might do is lower the bar and allow people who couldn't otherwise get into the field to get in and actually write some piece of usable computer program.
So if you look at the actual ransomware executables, they haven't changed very much in several years because they do everything that the crooks need them to do.
So it's very unlikely that AI is going to come along.
And this is maybe we haven't seen an explosion of AI in cybersecurity, certainly not amongst cybercriminals, because at the moment they just don't seem to need it.
So ransomware does what it needs to do. So you're not going to get an AI come along and write a better ransomware because it's not going to get much better than it is.
But what it might do is lower the bar and allow people who couldn't otherwise get into the field to get in and actually write some piece of usable computer program.
And maybe also increase the scale of it. So at the moment, we see things like pig butchering scams and romance scams which are going on.
And I can imagine that people could, you know, a gang could perhaps use AI to target many, many, many thousands more people at the same time and have their AIs producing the responses to the messages which this person who's fallen in love with them is giving them.
I can imagine that happening, maybe not so far down the track.
And I can imagine that people could, you know, a gang could perhaps use AI to target many, many, many thousands more people at the same time and have their AIs producing the responses to the messages which this person who's fallen in love with them is giving them.
I can imagine that happening, maybe not so far down the track.
Well, we're kind of fucked, I think. So this has been a really fun episode.
I don't even have good news at the end of this other than to say maybe it's high time we take a page of Socrates book according to Plato and basically say the true wise recognize that they know absolutely F-A.
And I'm becoming a serious wise-ass, guys.
I don't even have good news at the end of this other than to say maybe it's high time we take a page of Socrates book according to Plato and basically say the true wise recognize that they know absolutely F-A.
And I'm becoming a serious wise-ass, guys.
That's very, very deep, Carole.
So what you're saying is that Socrates said, I think we've gone too far.
This episode of Smashing Security is sponsored by Kolide.
Wouldn't it be great if a device which lacked compliance or lacked security was denied access to your organization's SaaS apps and other resources?
Because this would mean that the hackers who had nabbed the unlucky employee's credentials, for example, could not gain access to your assets. It would effectively lock them out.
Welcome to Kolide, a world where access is only given to approved secure devices. As the administrator, you can manage every operating system, even Linux, from a single dashboard.
Another bonus of Kolide: employees can often fix their own problems without involving IT support, meaning less resources are needed to effectively operate a more secure environment.
Kolide is the device trust solution for companies with Okta. Kolide ensures that if a device is not trusted or it's insecure, it is denied access to your cloud apps.
Learn more at kolide.com/smashing. That's k-o-l-i-d-e.com/smashing. And huge thank you to Kolide for sponsoring the show.
Wouldn't it be great if a device which lacked compliance or lacked security was denied access to your organization's SaaS apps and other resources?
Because this would mean that the hackers who had nabbed the unlucky employee's credentials, for example, could not gain access to your assets. It would effectively lock them out.
Welcome to Kolide, a world where access is only given to approved secure devices. As the administrator, you can manage every operating system, even Linux, from a single dashboard.
Another bonus of Kolide: employees can often fix their own problems without involving IT support, meaning less resources are needed to effectively operate a more secure environment.
Kolide is the device trust solution for companies with Okta. Kolide ensures that if a device is not trusted or it's insecure, it is denied access to your cloud apps.
Learn more at kolide.com/smashing. That's k-o-l-i-d-e.com/smashing. And huge thank you to Kolide for sponsoring the show.
Shortcut compliance without shortchanging security. That's what Vanta can bring your company.
Expanding the scope of your security program with Vanta's market-leading compliance automation, saving your business time and money.
Vanta has over 5,000 customers around the globe who are saving over 300 hours in manual work and up to 85% of their costs for SOC 2, ISO 27001, HIPAA, GDPR, custom frameworks, and more.
And with Vanta's 200+ integrations, you can easily monitor and secure the tools your business relies on.
From the most in-demand frameworks to third-party risk management and security questionnaires, Vanta gives SaaS businesses of all sizes one place to manage risk and prove security in real time.
And as a special bonus, Smashing Security listeners can get a stonking 20% off Vanta. Just go to vanta.com/smashing to claim your discount. That's vanta.com/smashing.
And thanks to Vanta for supporting the show. And welcome back. And you join us at our favorite part of the show, the part of the show that we like to call Pick of the Week.
Expanding the scope of your security program with Vanta's market-leading compliance automation, saving your business time and money.
Vanta has over 5,000 customers around the globe who are saving over 300 hours in manual work and up to 85% of their costs for SOC 2, ISO 27001, HIPAA, GDPR, custom frameworks, and more.
And with Vanta's 200+ integrations, you can easily monitor and secure the tools your business relies on.
From the most in-demand frameworks to third-party risk management and security questionnaires, Vanta gives SaaS businesses of all sizes one place to manage risk and prove security in real time.
And as a special bonus, Smashing Security listeners can get a stonking 20% off Vanta. Just go to vanta.com/smashing to claim your discount. That's vanta.com/smashing.
And thanks to Vanta for supporting the show. And welcome back. And you join us at our favorite part of the show, the part of the show that we like to call Pick of the Week.
Pick of the Week.
Pick of the Week.
Pick of the Week is the part of the show where everyone chooses something they like.
Could be a funny story, a book that they've read, a TV show, a movie, a record, a podcast, a website, or an app, whatever they like.
It doesn't have to be security-related necessarily. Better not be. Well, my Pick of the Week this week is security-related, and I'm not ashamed to say it.
And my choice this week comes about because I was contacted by a loyal listener to the show, Alan Liska, and he told me about a radio drama series which used to be on in the 1940s and early, all the way through to the early '60s, actually, called Yours Truly, Johnny Dollar.
Oh, never heard of it. It was the adventures, the adventures of a private insurance investigator with an action-packed expense account. That's how it was promoted.
That's what people did in the old days, folks, for their entertainment.
Well, the tales of this private insurance investigator, it's now all— the character's fallen into the public domain, which has meant that Alan and some of his buddies have been hard at work updating Johnny Dollar and bringing him to the present day.
And they have made a series of comic books where the private investigator is now a cybersecurity insurance investigator, still with an action-packed expense account.
I was surprised as I was reading it just how often he wrote down his expenses for taking cabs or buying a new hat or taking a receptionist out for lunch in order to get some information from her, that kind of thing.
So Alan has put this together. He is selling it on his website and they've also about to launch a Kickstarter for their third issue.
And I checked it out and I thought, oh, you know, this is a bit of fun and it's cybersecurity related, which I know we love to have our pick of the weeks cybersecurity related.
So I thought I'd give it a mention. So you can find it at JohnnyDollar.io if your interest has been piqued. And if you're a fan of comic books, that is where you should go.
Could be a funny story, a book that they've read, a TV show, a movie, a record, a podcast, a website, or an app, whatever they like.
It doesn't have to be security-related necessarily. Better not be. Well, my Pick of the Week this week is security-related, and I'm not ashamed to say it.
And my choice this week comes about because I was contacted by a loyal listener to the show, Alan Liska, and he told me about a radio drama series which used to be on in the 1940s and early, all the way through to the early '60s, actually, called Yours Truly, Johnny Dollar.
Oh, never heard of it. It was the adventures, the adventures of a private insurance investigator with an action-packed expense account. That's how it was promoted.
That's what people did in the old days, folks, for their entertainment.
Well, the tales of this private insurance investigator, it's now all— the character's fallen into the public domain, which has meant that Alan and some of his buddies have been hard at work updating Johnny Dollar and bringing him to the present day.
And they have made a series of comic books where the private investigator is now a cybersecurity insurance investigator, still with an action-packed expense account.
I was surprised as I was reading it just how often he wrote down his expenses for taking cabs or buying a new hat or taking a receptionist out for lunch in order to get some information from her, that kind of thing.
So Alan has put this together. He is selling it on his website and they've also about to launch a Kickstarter for their third issue.
And I checked it out and I thought, oh, you know, this is a bit of fun and it's cybersecurity related, which I know we love to have our pick of the weeks cybersecurity related.
So I thought I'd give it a mention. So you can find it at JohnnyDollar.io if your interest has been piqued. And if you're a fan of comic books, that is where you should go.
Cool. Thanks, Alan.
Thank you, Alan, for listening. Mark, what's your pick of the week?
So mine is a website. So I like to cook things because, you know, I like to stay alive, so I have to eat.
And over the years, I have learned how to make food tastier than I could when I was younger. And so I spend a lot of— I use a lot of recipes from the web.
And I normally don't care where they come from. Normally, I'm just like, I'm going to make a thing, like I want to do some Japanese fish or something like that.
And I'll just Google it and a recipe will come up. And sometimes they are really good, and sometimes they are not really good.
And they always have that great long life story at the beginning, the SEO blurb that everybody has to put in to pad out there. Could surely everyone could just agree to scrap that?
Like, Google, what are you doing? Just like, if it's got recipe on the page, just ignore everything before that, please. I know, I agree, it's so painful.
And so over time, you kind of develop ideas of which websites are good and which websites are less good.
And so when a recipe comes up, when I'm looking for something and a recipe comes up on a site that I recognize, I go, oh, you know, maybe I'll pick that one because that's a good site.
But I don't have websites where I go specifically. I'll go to that website and I'll pick one of the recipes from that site because it's so good, with one exception.
And the exception is a website called I Heart Umami. And I stumbled upon it looking for a satay chicken recipe, and I liked it so much.
I was looking for a satay chicken I could do in an air fryer, and I liked it so much that I went back there and I started cooking other things that were on this website.
And I've done a number of dishes, like different salads and a number of dishes. All have been really, really good.
So if you want to come off as a better cook than you are, then I cannot recommend this website. Graham's like, check.
I cannot recommend this website highly enough because it will absolutely make your food taste like the best version of your food.
And it's also, it's all low carb, gluten free, which I don't care about that at all. I love gluten and I love carbs.
But, you know, if that's important to you, it's all that, and it's kind of pretty keto-friendly as well.
So it's a really, really lovely kind of protein-rich, flavorful dishes from the East.
And over the years, I have learned how to make food tastier than I could when I was younger. And so I spend a lot of— I use a lot of recipes from the web.
And I normally don't care where they come from. Normally, I'm just like, I'm going to make a thing, like I want to do some Japanese fish or something like that.
And I'll just Google it and a recipe will come up. And sometimes they are really good, and sometimes they are not really good.
And they always have that great long life story at the beginning, the SEO blurb that everybody has to put in to pad out there. Could surely everyone could just agree to scrap that?
Like, Google, what are you doing? Just like, if it's got recipe on the page, just ignore everything before that, please. I know, I agree, it's so painful.
And so over time, you kind of develop ideas of which websites are good and which websites are less good.
And so when a recipe comes up, when I'm looking for something and a recipe comes up on a site that I recognize, I go, oh, you know, maybe I'll pick that one because that's a good site.
But I don't have websites where I go specifically. I'll go to that website and I'll pick one of the recipes from that site because it's so good, with one exception.
And the exception is a website called I Heart Umami. And I stumbled upon it looking for a satay chicken recipe, and I liked it so much.
I was looking for a satay chicken I could do in an air fryer, and I liked it so much that I went back there and I started cooking other things that were on this website.
And I've done a number of dishes, like different salads and a number of dishes. All have been really, really good.
So if you want to come off as a better cook than you are, then I cannot recommend this website. Graham's like, check.
I cannot recommend this website highly enough because it will absolutely make your food taste like the best version of your food.
And it's also, it's all low carb, gluten free, which I don't care about that at all. I love gluten and I love carbs.
But, you know, if that's important to you, it's all that, and it's kind of pretty keto-friendly as well.
So it's a really, really lovely kind of protein-rich, flavorful dishes from the East.
And the recipes are approachable? Like, it's not too complicated or too expensive with crazy ingredients?
Yes, yes. So this is not an Ottolenghi cookbook.
I made the mistake— somebody bought me an Ottolenghi book for Christmas a couple of years ago, and I made the mistake— it was the simple book.
I made the mistake of cooking one thing from it, and it took me 2 hours and it had 20 ingredients, and I thought, this is not a simple recipe. This is not for me.
But yeah, no, it's really good, kind of hearty, soulful, fairly straightforward. But you can make it complicated. But yeah, it's great. Check it out. So I Heart Umami.
I made the mistake— somebody bought me an Ottolenghi book for Christmas a couple of years ago, and I made the mistake— it was the simple book.
I made the mistake of cooking one thing from it, and it took me 2 hours and it had 20 ingredients, and I thought, this is not a simple recipe. This is not for me.
But yeah, no, it's really good, kind of hearty, soulful, fairly straightforward. But you can make it complicated. But yeah, it's great. Check it out. So I Heart Umami.
I'm on the website right now. It looks very straightforward. Food looks delicious. And yeah, looks interesting. Nice one, Mark. I will give it a go. Carole, what's your pick of the week?
As many of you know, I power through a lot of audiobooks, but the thing is, it gets expensive, right? Especially if you're sick with the flu.
So this past weekend, you know, I had an audiobook humming away in the background 'cause I was super sick. I was stuck in bed.
And I had this audiobook playing, and then it's done, and I kind of heard it, but I didn't hear it completely. And, you know, I kind of want another one.
It just gets expensive or whatever. So I'm powering through these books, and I see the hit on the bank account. And that's what got me off my butt.
And I got down to the local library and joined the local library, which is kind of sad that I haven't done that before.
You guys are probably both members of libraries, probably because of kids.
So this past weekend, you know, I had an audiobook humming away in the background 'cause I was super sick. I was stuck in bed.
And I had this audiobook playing, and then it's done, and I kind of heard it, but I didn't hear it completely. And, you know, I kind of want another one.
It just gets expensive or whatever. So I'm powering through these books, and I see the hit on the bank account. And that's what got me off my butt.
And I got down to the local library and joined the local library, which is kind of sad that I haven't done that before.
You guys are probably both members of libraries, probably because of kids.
That does tend to encourage it, but yes, save a lot of money.
Yeah, tell me about it. So it's— I've been to the one in the center of Oxford, but I haven't been to my local one, which is a kilometer away from my place.
And it's a beautiful building, you know, lovely knowledgeable people. But as part of your library access, you also have access to their audiobook selection for free.
I've been using the Libby app. I don't know if either of you've ever used it. No.
So it's tied with libraries and my friend, it seems to be an international library app 'cause I have a friend in California who uses the same app for his public library.
And the app is stable, it's easy to use, it's not too flashy. So from a usability point of view, I've been using it for about 2 months now. I think it's pretty solid.
And you can put holds on books, you can renew if you need to, you can return early, all that stuff.
The only thing that's a bit shitty is the search function because there's a lot of stuff in libraries that may not be for everyone.
Our library seems to have a huge fantasy romance section, which is not my area or my bag, right?
But it's difficult to remove them from searches, so they kind of crop up everywhere, basically. But that said, I'm loving it.
And I've just finished Last Night in Montreal by Emily St. John Mandel, an amazing book. And I've started a new one, a classic, The Bell by Iris Murdoch.
All brilliant lessons, all free, and I'm showing my support for my local library. So that is my pick of the week, libraries. Yeah, bravo.
And it's a beautiful building, you know, lovely knowledgeable people. But as part of your library access, you also have access to their audiobook selection for free.
I've been using the Libby app. I don't know if either of you've ever used it. No.
So it's tied with libraries and my friend, it seems to be an international library app 'cause I have a friend in California who uses the same app for his public library.
And the app is stable, it's easy to use, it's not too flashy. So from a usability point of view, I've been using it for about 2 months now. I think it's pretty solid.
And you can put holds on books, you can renew if you need to, you can return early, all that stuff.
The only thing that's a bit shitty is the search function because there's a lot of stuff in libraries that may not be for everyone.
Our library seems to have a huge fantasy romance section, which is not my area or my bag, right?
But it's difficult to remove them from searches, so they kind of crop up everywhere, basically. But that said, I'm loving it.
And I've just finished Last Night in Montreal by Emily St. John Mandel, an amazing book. And I've started a new one, a classic, The Bell by Iris Murdoch.
All brilliant lessons, all free, and I'm showing my support for my local library. So that is my pick of the week, libraries. Yeah, bravo.
Good for you. And libraries aren't just good for books and audiobooks and things like that, but there's an awful lot of other good stuff that goes on them.
For instance, if you go to your local library, you might find that they have special events for parents with young children.
So they might have a Lego club or, you know, rhyme time, you know, somewhere to dump your kids for an hour or so. You can get them involved and interested in library there.
And also, for people who struggle with technology, if you have relatives or if listeners have trouble with some of the computer issues, sometimes they have digital help sessions at a library as well, where they'll help you sort out your iPad or whatever it is that you're listening to the podcast on.
Make sure that you've got it all configured right.
For instance, if you go to your local library, you might find that they have special events for parents with young children.
So they might have a Lego club or, you know, rhyme time, you know, somewhere to dump your kids for an hour or so. You can get them involved and interested in library there.
And also, for people who struggle with technology, if you have relatives or if listeners have trouble with some of the computer issues, sometimes they have digital help sessions at a library as well, where they'll help you sort out your iPad or whatever it is that you're listening to the podcast on.
Make sure that you've got it all configured right.
And I've been to my library a few times since, and it's so quiet and it's a beautiful building and there's not a lot of people in there. So you can actually just get some work done.
And it's quite cool. So if you need a quiet space, think about it, libraries.
And it's quite cool. So if you need a quiet space, think about it, libraries.
Often free Wi-Fi as well. If you want to set up some kind of criminal, cybercriminal enterprise, it's a good place to do it from. Just a thought. Just a thought. Right.
Well, that just about wraps up the show for this week. Mark, I'm sure lots of our listeners would love to follow you online, find out what you're up to.
What is the best way for folks to do that?
Well, that just about wraps up the show for this week. Mark, I'm sure lots of our listeners would love to follow you online, find out what you're up to.
What is the best way for folks to do that?
You can find me on X. Come on.
Don't do that.
You can find me on Twitter @MarkStockley. Better.
And you can follow us on Twitter @SmashingSecurity, no G. Twitter doesn't allow us to have a G. We also have a Mastodon account.
And don't forget to ensure you never miss another episode. Follow Smashing Security in your favorite podcast app, such as Apple Podcasts, Spotify, and Overcast.
And don't forget to ensure you never miss another episode. Follow Smashing Security in your favorite podcast app, such as Apple Podcasts, Spotify, and Overcast.
And a billion thank yous to our episode sponsors, Kolide, and Vanta, and of course to our wonderful Patreon community. It's thanks to them all that this show is free.
For episode show notes, sponsorship info, guest list, and the entire back catalog of more than — 356 episodes, check out smashingsecurity.com. Until next time, cheerio.
For episode show notes, sponsorship info, guest list, and the entire back catalog of more than — 356 episodes, check out smashingsecurity.com. Until next time, cheerio.
Bye-bye. Bye. Bye-bye.
One day we're going to be saying 500 times. Are we? Yeah. Jesus Christ. Well, unless you die. Are you dying?
No. Possibly. Don't die yet. Okay, hang on.
Let's get more material so the AI can fake you and be a fake host. And I can crack on as normal and no one will be able to tell the difference.
Hosts:
Graham Cluley:
@grahamcluley.com
@
/ grahamcluley
Carole Theriault:
Guest:
Mark Stockley:
Episode links:
- Mobile phone stolen every six minutes in London, says Met Police – BBC News.
- iPhone Thief Explains How He Breaks Into Your Phone – YouTube.
- About Stolen Device Protection for iPhone – Apple.
- Cops Used DNA to Predict a Suspect’s Face—and Tried to Run Facial Recognition on It – Wired.
- Will ChatGPT write ransomware? Yes – Malwarebytes.
- AI chatbots are making scams more convincing than ever, warn spy chiefs – The Telegraph.
- Test yourself: which faces were made by AI? – New York Times.
- AI vs. Human Writing: Experts Fooled Almost 62% of the Time– Neuroscience News.
- I know that I know nothing – Wikipedia.
- Yours truly, Johnny Dollar – Comic book.
- I Heart Umami.
- Libby.
- Smashing Security merchandise (t-shirts, mugs, stickers and stuff)
Sponsored by:
- Kolide – Kolide ensures that if your device isn’t secure it can’t access your cloud apps. It’s Device Trust for Okta. Watch the demo today!
- Vanta – Expand the scope of your security program with market-leading compliance automation… while saving time and money. Smashing Security listeners get 10% off!
Support the show:
You can help the podcast by telling your friends and colleagues about “Smashing Security”, and leaving us a review on Apple Podcasts or Podchaser.
Become a supporter via Patreon or Apple Podcasts for ad-free episodes and our early-release feed!
Follow us:
Follow the show on Bluesky at @smashingsecurity.com, or on Mastodon, on the Smashing Security subreddit, or visit our website for more episodes.
Thanks:
Theme tune: “Vinyl Memories” by Mikael Manvelyan.
Assorted sound effects: AudioBlocks.
As ALWAYS, another great episode! Thank you for such a great podcast!!