If you were in your kitchen talking about Thailand, then the next time you went online on your mobile an advert came up about Thailand, you'd think it was coincidence. If you'd just been told that a family friend had been killed in a road accident in Thailand and the next time you used a search engine on your phone, up popped the name of the friend, and the words, "Motorbike accident, Thailand" and the year in the suggested text below the search box – you might well think your phone had been listening to your conversation.
And if you were the BBC, and told the story above, along with increasing numbers of viewers with similar stories, suggesting that their phones had been listening to them then targeting them with adverts related to the things they'd been talking about, you might wonder if you were in X-files territory. Or you might decide to investigate anyway to see if it was even possible. Which is just what the BBC did.
Pen Test Partners' Ken Muro was called in to investigate whether it was possible that advertisers are using audio for targeting purposes following what were described as numerous anecdotes from users who claimed to have received adverts related to conversations they had had on their mobiles.
Munro told SCMagazineUK.com that he was a little sceptical that it could be happening saying: “We didn't think it would be so straightforward and that the battery would drain so little, as the phone was doing online processing, and we thought that continually posting audio data to a ‘voice to text' service would be battery intensive and be noticed, but it wasn't. And we didn't realise how good the voice recognition would be, and how much we'd pick up.”
First Munro and his team concluded that users rarely check the permissions they give away when accepting terms and conditions as they install an app. Therefore they decided that it would be easy to get an app on to a user's phone that could listen.
David Lodge at Pen Test Partners wrote a mobile app, whose terms and conditions included permissions to record, installed it in the researchers' phone, and hooked up to a third party processing voice-totext service, which then sent that text to another third party – Pen Test Partners. The text results were presented in real time on screen. Muro said, “Anyone with a modicum of Android or iOS coding skills could have done this, demonstrating that it is perfectly possible that numerous mobile apps could snoop on conversations.” If the phone had a data connection – broadband, WiFi or whatever, it would listen.
The user had to download, and install the app, and the user had to accept permissions – and most don't review permissions. Munro noted how Facebook, Twitter, Instagram and many other social media mobile apps already give themselves ‘record audio' permission, adding: “No-one is saying that they use it, but microphone access is widespread. It's very common and people don't understand the implications.”
Although iOS could have been used in the same way, the test app was carried out on Android as it would have been easier to get onto its apps store, though this one was not put into any online store, “We downloaded to our device – we are not putting rogue apps into the wild,” confirmed Munro.
There were some caveats, however. The media stream of the phone had to be muted, to avoid it making sounds whilst recording, and while keywords were set to try to generate custom adverts within the app, they didn't actually work.
Munro said that it would be difficult to determine if it really is happening in the wild, as some of the claims came from tablet users, some from phones, and there are an increasing number of TVs running these apps. He noted how TVs may be listening to you in the future – as its not much of a stretch to use them to deliver customised adverts or just snoop on people.
Both Munro and the BBC agreed that it could still be coincidence – or could just have been based on other things, such as people picking up on references to things they have been discussing. It could be the result of a third party developer who left code intentionally or unintentionally.
However, if an app was snooping on you, it would be very difficult to determine without looking on the devices. The next step would be figuring out a way to review large numbers of apps in the stores to see if any are actually taking your voice data. You could check your device for forensic investigation so it would be possible to find the rogue code – and Ken Munro welcomes the opportunity to check your device if you believe you are a victim.
Is victim the right word though? After all, what law is broken if you've given permission in the terms and conditions – but who would contend that this was informed consent?