Deepfake video of Taylor Swift speaking in Mandarin goes viral in China, sparks discussion on AI

American pop star Taylor Swift got the attention of her Chinese fans this week when clips of her speaking fluent Mandarin went viral on Chinese social media.

In a video posted on Chinese social media platform Weibo, the 33-year-old singer can be seen flaunting her Mandarin-speaking skills in what looks to be a talk show.

“Recently, I’ve been to many places, like Italy, France and Japan,” Swift said in Mandarin with a slight American twang. The video was shared on Oct 21 and has since gained over six million views.

In another video, Swift talked about songs that are “left behind”, and how she wished people could hear these tracks.

The twist? Swift owes her Mandarin-speaking prowess to artificial intelligence (AI).

The deepfake videos were generated by an AI tool from HeyGen, a Chinese start-up, reported news outlet The China Project.

The lip movements were synced to make it look like she was speaking in Mandarin, and it sounded like Swift’s voice, making the videos so realistic that it led Chinese netizens to express their amazement at the technology, with comments like: “This is awesome!”

Others asked for Mandarin versions of their favourite shows.

But it also triggered discussion over the potential pitfalls that will come as AI gets more advanced, like scams and the loss of jobs.

“The scariest thing is that if someone uses his technology to make fake news, because the AI is able to manipulate the voice and movement of the mouth, people will easily believe it,” said one netizen.

Another Weibo user wrote that it would be “terrible if (this AI technology) is used for fraud”, as criminals can use it to dupe victims for money.

HeyGen, the company behind the tool that made the videos, was co-founded in November 2020 by Joshua Xu and Wayne Liang.

The firm’s AI-powered video generator allows users to create text-to-speech videos in over 300 voices in more than 40 languages, using over 100 AI avatars with different ethnicities, ages, poses and clothes.

Technology like this has sparked concern over the criminal applications of AI. Earlier in 2023, a scammer in northern China used deepfake technology to convince a man to transfer money to a supposed friend.

The scammer used AI-powered face-swopping technology to impersonate a friend of the victim during a video call. The man ended up transferring him some 4.3 million yuan (S$823,000).

In January, China put in force new rules that require businesses offering deepfake services to obtain the real identities of their users.

It also requires deepfake content to be appropriately tagged to avoid confusion.