Scientists prove AI can fool deepfake detectors in ‘real-world threat’ for first time

0
14
computer


For the very first time, scientists have proven that artificial intelligence can deceive systems designed to detect deepfakes.

A portmanteau of “deep learning” and “fake”, deepfakes are videos designed to fool the viewer by merging real footage with artificially generated images.

While some might be funny, such as manipulating a politician’s mouth so it looks as though they’re saying something absurd, the technology is advancing rapidly and experts are concerned it will make damaging disinformation even more convincing.

Systems have been designed to detect when footage is a deepfake, but those systems can be fooled, researchers from the University of California San Diego revealed at a conference.

Deepfakes are of increasing concern to cybersecurity experts

Typical deepfake detectors focus on faces in videos they’re analysing, tracking it and passing the data on to a neural network that determines whether the face is real or not.

Detectors will often focus on unnatural blinking as deepfakes don’t tend to replicate the eye movement convincingly.

Speaking at the Workshop on Applications of Computer vision conference, held online in January, scientists demonstrated how detectors can be tricked by inserting inputs called “adversarial examples” into every frame of a deepfake video.

President Barack Obama
Former President Barack Obama is a typical target of deepfake videos

These examples are slightly manipulated inputs that cause AI systems, such as machine learning models, to make a mistake — and they work even after videos are compressed.

The most advanced deepfake detectors rely on these machine learning models to function, so the news is troubling, to say the least.

“Our work shows that attacks on deepfake detectors could be a real-world threat,” computer engineering PhD student Shehzeen Hussain told the conference.

queen
A deepfake of the Queen went viral last year that showed Her Majesty warning of the technology’s dangers

“More alarmingly, we demonstrate that it’s possible to craft robust adversarial deepfakes in even when an adversary may not be aware of the inner workings of the machine learning model used by the detector.”

Computer science student Paarth Neekhara added: “If the attackers have some knowledge of the detection system, they can design inputs to target the blind spots of the detector and bypass it.”

The researchers created an adversarial example for every face in a video frame, built to withstand compressing and resizing which will usually remove them.

The modified version of the face is then inserted into every frame.

computer keyboard
Systems designed to detect deepfakes are fallible and can be fooled

The study authors tested their deepfake attacks in two different scenarios. In one, attackers had full access to the detector model, including the face extraction pipeline and the architecture and parameters of the classification model.

In the other, the attackers could only query the machine learning model to work out the probabilities of a frame being classified as real or fake.

The attack’s success rate in the first scenario was 99% for uncompressed videos and 84.96% for compressed.

In the second scenario, the success rate was 86.43% for uncompressed and 78.33% for compressed videos.

computer
Researchers say such systems can be “easily bypassed”

It’s the first work that demonstrates successful attacks on state-of-the-art deepfake detectors.

“To use these deepfake detectors in practice, we argue that it is essential to evaluate them against an adaptive adversary who is aware of these defences and is intentionally trying to foil these defences,”‘ the researchers said.

“We show that the current state of the art methods for deepfake detection can be easily bypassed if the adversary has complete or even partial knowledge of the detector.”





Source link

LEAVE A REPLY

Please enter your comment!
Please enter your name here