The history of deep learning goes back as far as 1943, when Walter Pitts and Warren McCulloch created a computer model based on the neural networks of the human brain
Today, if we asked a language model like GPT-3 to write an article about the history of deep learning, it might begin with that sentence.
Many changes led from Pitts and McCulloch’s early neural network to what we now call “deep learning”: the addition of backpropagation (Yann LeCun and others), and the creation of “deep” networks with many “hidden layers” (Geoff Hinton and others) are perhaps the most important. And while early neural networks couldn’t be programmed effectively (if at all) on the computers of their day, deep learning has now become commonplace.
Now, large language models that use deep learning have become an integral cog in the fake news machine, which drives propaganda, false information and, to the naked, untrained eye, almost undetectable impersonations of real, influential people. As with almost any tool, asking whether deep learning is a force for good or evil is pointless. It has already contributed a lot of good. Equally undoubtedly, it will be used for evil.
Let’s start with the benefits that natural language technology can provide. Automated translation is one way for NLP to contribute to the public good. Imagine living in a country, such as India, where there are 22 official languages. Similarly, Bonaventure Dossou was keen to improve phone conversations with his mother, who would often send him voice messages in Fon, a Beninese language. Bonaventure often couldn’t understand some of the phrases his mother used, which made conversation difficult. To overcome this, he created an artificial intelligence language translation model, powered by deep learning, often used by modern NLP algorithms. The model is still a work in progress, but it shows how real-time translation can allow people who speak two completely different languages to communicate with each other.
Outside of translation, deep learning-powered NLP can be used to ease the workload of people in industries like journalism. Articles such as financial reports, which few journalists enjoy writing, can and, in many cases, are being written automatically through the use of deep learning technology. Rather than writing these reports, journalists can spend more time working on important research and writing the more creative, in-depth articles they prefer.
There is also a dark side. In the form of fake news, deep learning is beginning to cast a rather large shadow. In March 2021, a landmark vote was scheduled to decide whether the first ever labour union should be created at a US-based Amazon warehouse. Leading up to the vote, multiple deepfake Twitter accounts were created, using convincing profile pictures and posts to defend Amazon’s working practices. These fake accounts were probably pranks, rather than an effort by Amazon itself; but regardless of the motivation, they could have been the work of “deepfake” language technology. (The accounts included profile pictures that had many of the characteristic flaws of deepfake images.) In this instance, the Tweets had little impact other than to cause confusion over who was responsible. However, it was a worrying reminder of how easily a false narrative can be created and perpetuated using deep learning technology.
Even more concerning, the technology is becoming less expensive and more accessible. At current rates, creating 8.5 million 20-word comments on Twitter would only cost around £6,000. And, in many instances, you wouldn’t even need half that many Tweets to spread a narrative. Imagine, though, if someone put real money behind this and the impact that could have on elections, public health communication, and market trends? How do we combat this and ensure that deep learning technology is being used for good rather than bad?
Work has gone into counteracting the negative implications of deep learning and its role in the spread of fake news. For example, one algorithm can detect fake tweets with about 90% accuracy. Another solution relies on whether a text model can predict the next word in an article or post. The more predictable the content is, the more likely it was written by a machine.
It’s certainly encouraging that these solutions are being developed. However, there’s going to be a constant game of cat and mouse between those generating fake news and the people looking to put a stop to it. The deep learning technologies used to create the content will be updated to avoid detection, putting the ball firmly back into the court of social media providers and regulators to find new ways to detect and remove it.
The onus is on social media providers to find a solution to the spread of fake news on their platforms. Some of them don’t have the research power to put into the fight. Therefore, it could come down to whether a technology giant such as Google or Facebook is willing to put the time and investment into putting a stop to it. It will require consistent investment and research into systems and solutions which can quickly identify content created by a deep learning model.
About the Author
Mike Loukides is Vice President of Emerging Tech at O’Reilly. Inspiring the future for more than 40 years We share the knowledge and teach the skills people need to change their world. For more than 40 years, O’Reilly has imparted the world-shaping ideas of innovators through books, articles, conferences, and our online learning platform. When individuals, teams, and entire enterprises connect with the world’s leading experts and content providers, anything is possible. Whether you’re working to advance your career, be a better manager, or achieve the next breakthrough in technology or business, learning new skills is at the heart of it all.
