If AI and deep fakes can listen to a video or audio of a person and then are able to successfully reproduce such person, what does this entail for trials?
It used to be that recording audio or video would give strong information which often would weigh more than witnesses, but soon enough perfect forgery could enter the courtroom just as it’s doing in social media (where you’re not sworn to tell the truth, though the consequences are real)
I know fake information is a problem everywhere, but I started wondering what will happen when it creeps in testimonies.
How will we defend ourselves, while still using real videos or audios as proof? Or are we just doomed?
Fake evidence, e.g. forged documents, are not not new things. They take things like origin, chain of custody etc into account.
As someone who works in the field of criminal law (in Europe, and I would be shocked if it wasn’t the same in the US) - I’m not actually very worried about this. By that I don’t mean to say it’s not a problem, though.
The risk of evidence being tampered with or outright falsified is something that already exists, and we know how to deal with it. What AI will do is lower the barrier for technical knowledge needed to do it, making the practice more common.
While it’s pretty easy for most AI images to be spotted by anyone with some familiarity with them, they’re only going to get better and I don’t imagine it will take very long before they’re so good the average person can’t tell.
In my opinion this will be dealt with via two mechanisms:
-
Automated analysis of all digital evidence for signatures of AI as a standard practice. Whoever can be the first person to land contracts with police departments to provide bespoke software for quick forensic AI detection is going to make a lot of money.
-
A growth in demand for digital forensics experts who can provide evidence on whether something is AI generated. I wouldn’t expect them to be consulted on all cases with digital evidence, but for it to become standard practice where the defence raises a challenge about a specific piece of evidence during trial.
Other than that, I don’t think the current state of affairs when it comes to doctored evidence will particularly change. As I say, it’s not a new phenomenon, so countries already have the legal and procedural framework in place to deal with it. It just needs to be adjusted where needed to accommodate AI.
What concerns me much more than the issue you raise is the emergence of activities which are uniquely AI dependent and need legislating for. For example, how does AI generated porn of real people fit into existing legislation on sex offences? Should it be an offence? Should it be treated differently to drawing porn of someone by hand? Would this include manually created digital images without the use of AI? If it’s not decided to be illegal generally, what about when it depicts a child? Is it the generation of the image that should be regulated, or the distribution? That’s just one example. What about AI enabled fraud? That’s a whole can of worms in itself, legally speaking. These are questions that in my opinion are beyond the remit of the courts and will require direction from central governments and fresh, tailor made legislation to deal with.
My bigger concern is the state using AI created fake data. It’s far harder to stop that, as false confessions and coerced confessions are already a problem. The process can’t really catch it, because it’s the people in charge of the process doing it.
-
I think other answers here are more essential - chain of custody, corroborating evidence, etc.
That said, Leica has released a camera that digitally signs its images, and other manufacturers are working on similar things. That will allow people to verify whether the image is original or has been edited. From what I understand Leica has some scheme where you can sign images when you update them too, so there’s a whole chain of documentation. Here’s a brief article
It’s an interesting experiment, but why would we trust everything that Leica supposedly verified? The same shit with digital signatures and blockchain stuff. We are at the gates of the world where we have zero trust by default and would only intentionally outsource verification to third parties we trust, because penalties for mistakes are growing each day.
I don’t think we should inherently. I’ve thought about the idea of digitally signed photos and it seems sound unless someone is quite clever with electronics. I’m guessing there’s some embedded key on the camera that is hard but maybe not impossible to access. If people can hack Teslas for “full autopilot” or run Doom on an ATM machine I’m not confident that this kind of encryption will never be cracked. However, I would hope an expert witness would also examine the camera that supposedly took the picture. I would think it to be impossible for someone to acquire the key without a 3rd party detecting the intrusion.
Today we have EXIFs and it’s better to wipe them all of these for privacy reasons. Because every picture you take otherwise contains a lot of your data like geoloc, model, exposuer, etc. That’s the angle they are yet to tackle - because most of these things are also leave us vulnerable.
They make Hardware Security Modules (HSMs) that are very difficult to crack, to the point that it is unbreakable at our current technology level. With a strong HSM, a high-bit per-device certificate signed by the company’s private key gives you authenticity and validation until the root key or HSM are broken, which is probably good enough for today while we try to figure out something better IMO.
Well as I said, I think there’s a collection of things we already use for judging what’s true, this would just be one more tool.
A cryptographic signature (in the original sense, not just the Bitcoin sense) means that only someone who possesses a certain digital key is able to sign something. In the case of a digitally signed photo, it verifies “hey I, key holder, am signing this file”. And if the file is edited, the signed document won’t match the tampered version.
Is it possible someone could hack and steal such a key? Yes. We see this with certificates for websites, where some bad actor is able to impersonate a trusted website. (And of course when NFT holders get their apes stolen)
But if something like that happened it’s a cause for investigation, and it leaves a trail which authorities could look into. Not perfect, but right now there’s not even a starting point for “did this image come from somewhere real?”
A camera that authenticates the timestamp and contents of an image is great. But it’s still limited. If I take that camera, mount it on a tripod, and take a perfect photograph of a poster of Van Gogh’s Starry Night, the resulting image will be yet another one of millions of similar copies, only with a digital signature proving that it was a newly created image today, in 2024.
Authenticating what the camera sensor sees is only part of the problem, when the camera can be shown fake stuff, too. Special effects have been around for decades, and practical effects are even older.
You’re right, cameras can be tricked. As Descartes pointed out there’s very little we can truly be sure of, besides that we ourselves exist. And I think deepfakes are going to be a pretty challenging development in being confident about lots of things.
I could imagine something like photographers with a news agency using cameras that generate cryptographically signed photos, to ward off claims that newsworthy events are fake. It would place a higher burden on naysayers, and it would also become a story in itself if it could be shown that a signed photo had been faked. It would become a cause for further investigation, it would threaten a news agency’s reputation.
Going further I think one way we might trust people we aren’t personally standing in front of would be a cryptographic circle of trust. I “sign” that I know and trust my close circle of friends and they all do the same. When someone posts something online, I could see “oh, this person is a second degree connection, that seems fairly likely to be true” vs “this is a really crazy story if true, but I have no second or third or fourth degree connections with them, needs further investigation.”
I’m not saying any of this will happen, just it’s potentially a way to deal with uncertainty from AI content.
Cameras with stronger security will become more and more important, though on a theoretical level, they could be cracked or forged, but I suppose it’s the usual cat and mouse game
Hardware signing stuff is not a real solution. It’s security through obscurity.
If someone has access to the hardware, they technically have access to the private key that the hardware uses to sign things.
A determined malicious actor could take that key and sign whatever they want to.
When video or audio evidence is submitted, it will be questioned as to its authenticity. Who recorded it? On what device? Then we’ll look for other corroborating evidence. Are there other videos that captured the events in the background of the evidence video? Are there witnesses? Is there contradictory evidence?
Say there’s a video depicting a person committing murder in an alley. The defense will look for video from the adjoining streets that show the presence or absence of the murderer before or after. If those videos show cars driving by with headlights on, they will look for corresponding changes in the luminosity of the crime video. If the crime happened in the daytime, they will check that the shadows correspond to Sun’s position at that moment. They’ll see if the reflections of objects match the scene. They’ll look for evidence that the murderer was not at the scene. Perhaps a neighbor’s surveillance camera shows they were at home or their cell phone indicated they were someplace else.
But if all these things indicate the suspect was in the alley and the video is legitimate, that’s powerful evidence toward a conviction.
Are there other videos that captured the events in the background of the evidence video?
I think this is key in a trial setting. A published picture might be unique but to think the photographer snapped just one picture while nobody else was present or also photographing is a bit of a stretch.
Science has proven that the entire model of human memory as factual testimony is a fallacy. That came out long before AI in the public space. I don’t think anyone has addressed that revelation. I doubt anyone will address this one. Hell, there are still people sketching the courtroom like cameras don’t exist. A president can stage a failed coup and a SC judge can fly the traitor’s flag and there are no consequences for either.
So what will be done, absolutely nothing, unless some billionaires stage a proxy war over it.
We’re not. Its going to upend our already laughably busted “justice” system to new unknown heights of cartoonish malfeasance.
Maybe each camera could have a unique private key that it could use to watermark keyframes with a hash of the frames themselves.
Usually I see non-technical people throw ideas like this and they’re stupid, but I’ve been thinking about this for a few minutes and it’s actually kinda smart
I think that’s exactly how it’s going to work - you can’t force all ‘fake’ sources to have signatures- it’s too easy to make one without one for malicious reasons. Instead you have to create trusted sources of real images. Much easier and more secure
One step closer to requiring smart phones to track an individual for their alibi
Disclaimer: I’m not an expert, just an interested amateur wanting to chat and drawing comparisons from past leaps in tech and other conversations/videos.
For a time expert analysis will probably work. For instance, the “click here to prove your not a robot” boxes can definitely be clicked by robots, but for now the robot moves in detectably different ways. My guess is that, for at least a while, AI content will be different from actual video in ways like code. There will probably be an arms race of sorts between AI and methods to detect AI.
Other forms of evidence like DNA, eyewitness accounts, cell phone tracking etc. will likely help mitigate deceitful AI somewhat. My guess is that soon video/audio will no longer be considered as ironclad as it was even a few years ago. Especially if it comes from an unverified source.
There are discussions about making AI tools have a digital “watermark” than can be used to identify AI-generated content. Of course this won’t help with black market-type programs, but it will keep most people out of the “deep fake for trials” game.
When it comes to misinformation on social media though, well…it’s probably going to get crazy. The last decade or so has been a race at an unprecedented scale to try and keep up with BS “proof”, psuedoscience, etc. Sadly those on the side of truth haven’t always won. The only answer I have for that is making sure people are educated about how to deal with misinformation and deepfakes - eg. awareness they exist, identifying reputable sources and expert consensus, and so on.
It’s a scary question, made a lot less scary by whoever it was that said “you know, I guess we’ve had text deepfakes a long time”
Eventually people just know it could be fake, so they look for other ways of verifying. The inevitability and the scale of it mean that, at the very least, we’ll have all our brainpower on it eventually.
It’s the meantime where shit could get wild.
