Update! (December 2019) CtrlShiftFace has a new video Home Stallone produced with Paperspace:
Deepfake technology is unlocking a new era of media production. As with all technologies, both positive and harmful use cases exist. As ethical technologists, we aspire to push the limits of what is possible, while minding the impact of the tools we create. We’re excited to see our users advance the state of the art in entertainment, media, and more.
Paperspace’s GPU cloud was originally designed to augment creative production through our CORE product. Now, with our Gradient suite of tools for ML & Data Science, and the advent of deepfake technology, we find ourselves sitting precisely at the nexus of the two. We are eager to see how CORE & Gradient get combined in innovative ways and have partnered with two deepfake producers leveraging both, in order to better understand how Paperspace helps their workflow and to assess the broader implications of the tech. We consider their viral face swap videos, (using high profile celebrities in equally famous source material) a benign way to highlight the sophistication of this technology, while encouraging the public to question the veracity of the content served to them.
Dr. Fakenstein and Ctrl_Shift_Face have been racking up view counts by the thousands (and sometimes millions) with their often hilarious, and frequently provocative mashups, taking remix culture of video content to a whole new level. “Dr. Fakenstein” is the online pseudonym for Jeff White, a now professional deepfake producer, living in New Zealand. “CtrlShiftFace” is the online persona of a European VFX specialist, who chooses to remain anonymous. After working with both to boost their creative output via Paperspace, we checked in to learn their stories, and to get their thoughts on the state of the art and how deep fakes are being received by the masses.
Who are you and what is your relationship to deep fakes?
CSF: n/a
DF: I do comedic deepfake videos for social media as @drfakenstein on Instagram. What originally started as a hobby now has me producing deepfake content professionally for TV, Film, Music Industry and Social Media with my company based in New Zealand.
Bring this project to life
How did you come across this tech?
CSF: I watched SIGGRAPH paper tech demos. That’s where I saw a deepfake for the first time, a few years ago. I love tech in all of its forms...and AI is the future of tech. It will be everywhere. I’m fascinated by it. I see deepfakes as a tool for creating these bizarre alternative timelines.
DF: I'm always learning about new tech. I had originally taught myself how to mine cryptocurrency and had excess computer GPU's when the bitcoin price took a nosedive in 2018. I read an article about deepfake technology and thought it may be entertaining to learn and make videos with. So I built a computer for deepfakes from crypto miner parts and experimented with swapping faces with deepfake software, I did them to make myself laugh originally and I thought other people may enjoy the videos them so posted them to Instagram.
What is the state of this tech? What are current challenges or limitations?
DF: It's still in its infancy and improving at a rapid rate, through software developments and techniques learned to improve the deepfake process. What I can do now compared to 6 months ago is a huge quality jump and it will only get better with time. Deepfakes go well with Paperspace. It allows me to render faces at a higher rate and quality than a typical deepfake setup could.
CSF: Well some SIGGRAPH papers on deepfakes from even a few years ago are much more advanced than what I use. The software I’m using is open source on GitHub. Elon Musk in A Space Odyssey had very challenging lighting. Also, closeups are difficult because the resolution of present deepfakes is not very high. It’s (typically) less than 200x200 pixels. The main limitation is computer power. You need high end, professional grade GPUs with massive amounts of VRAM, and that’s exactly what Paperspace provides.
How long until fake content is indistinguishable from real? Is the hype overblown or warranted?
DF: I think we are still years away from indistinguishable deepfakes. Even though “6 months” has been thrown around in the media, I believe it will be longer. I think some of the hype is overblown. There has been a lot of media attention recently around deepfake tech. While there is always risk of nefarious persons making content to frame people or spread false news, there are people working on the technology to detect these deepfakes. I also believe deepfakes for entertainment purposes are raising awareness of the technology and its capability.
CSF: Of course you could use it for evil stuff, but that’s not what I do. All faces are equal. You could even fake animated or video game characters on real people and vice versa. There is already tech from NVIDIA that can generate high res portraits. It’s just a matter of time before deepfakes will be so advanced that they’ll be indistinguishable from the real thing, but I think software for that won’t be readily available to the public but will more often be used by large budget movie studios.
In an ideal world, who has access to this technology?
DF: I don't believe it should be restricted, the deepfake software I use is open source. There are other deepfake technologies that have not been released by universities. It is just a matter of time before a company commercializes deepfake technology. Deepfakes are just another tool for making comedic videos, and the tech has allowed me to work with people I never imagined I'd work with, and to produce content that makes people laugh and brightens their day.
What do you think is missing from the current discussion around deep fakes?
DF: The potential in the entertainment industry gets overshadowed by the scaremongering. There are many different possibilities with deepfake technology that mostly get overlooked. Deepfakes for satire, movie "what if's", bringing back popular actors that have passed back into movies or TV, etc. Deepfake for stunt work in film and TV. Deepfake for the music industry/labels for artists who are always on the road or unreliable for music videos or content production. There is a negative stigma about deepfaking due to some people who initially used the technology for pornography by putting celebrity faces in them. There is a “Safe For Work” community of deepfakers who see the potential in the technology for entertainment purposes and putting out great content with a quickly growing audience.
CSF: The fear the press likes to spread around deepfakes is more dangerous than the actual deepfakes themselves. Again, it’s also important to mention that A.I. programs to detect deepfakes are being developed too, alongside deepfakes themselves.
What are some “worst case scenarios” involving deepfakes?
DF: That people believe that a deepfake is real and it ruins someone’s reputation or incriminates them.
CSF: Or the inverse, in which a powerful, political figure is actually caught doing something and can insist that the proof of their crime is fake, even if real.
