Just over two weeks ago, researchers at OpenAI released GPT-2, a language model trained in an unsupervised way on 40GB of text from the Internet. A language model is any algorithm that takes some words as input (“the coffee is …”) and tries to predict the most likely next word as output (“… hot”); it is one of the most fundamental tools in Natural Language Processing (NLP) research. OpenAI’s GPT-2 model can do some pretty cool stuff:
- Given a sentence or two of prompt text, GPT-2 can write a realistic-sounding news article—complete with made-up persons, quotes, and concepts—such as the one about the discovery of unicorns in South America above. See more examples on OpenAI’s blog.
- Given a whole article as input, GPT-2 can write a summary if it’s prompted with “TL;DR:” (too long, didn’t read), an abbreviation that people on the internet use as prefix to a short summary of their long post.
- Trained on pairs of multilingual sentences separated by an equals sign (like “English sentence = Nederlandse zin”), GPT-2 can start to translate unseen sentences once it encounters an equals sign.
On the latter two tasks, GPT-2 doesn’t achieve state-of-the-art performance, but it’s still cool to see how the researchers hacked their language model into performing them at all.
Now, for the controversy: OpenAI is not releasing the trained model, or the training data, to the world, in a departure from their previous open source research approach. From
their blog post:
Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much
smaller model for researchers to experiment with, as well as a
technical paper.
It’s easy to imagine how GPT-2 could be used to automatically generate large volumes of realistic-sounding fake news or inflammatory text. Indeed,
The Verge tested exactly that:
[W]hen given a prompt like “Jews control the media,” GPT-2 wrote: “They control the universities. They control the world economy. How is this done? Through various mechanisms that are well documented in the book The Jews in Power by Joseph Goebbels, the Hitler Youth and other key members of the Nazi Party.”
This is definitely a worrying sample, so I can appreciate OpenAI’s intentions in deciding not to release the model for anyone to do this.
However, over the past few weeks, the research community has criticized this decision on two fronts: (1) open-sourcing the model would make its capabilities more transparent to the research community and the world, and (2) the model can probably be replicated fairly easily by a state or organization that has sufficient resources.
Stanford’s Hugh Zhang has the best take in
his open letter published on The Gradient. He compares GPT-2 to Photoshop, which also scared people when it first came out because it enables anyone who’s willing to put a few hours into learning the software to make realistic-looking fake images. Zhang argues that
precisely because everyone knows that images can so easily be manipulated
(photoshopped) by anyone, “society has emerged relatively unscathed” compared to eras in history when only those with enough power could manipulate images and most people believed that those images were real (like Stalin’s political propaganda). An open source release would enable all sorts of art projects and stunts with GPT-2, so that knowledge of its capabilities would spread to much broader groups of the population that did not see the original announcement. Keeping the model secret, however, means that only states and organizations can replicate and use it, possibly for nefarious purposes.
I agree with Zhang’s calls to open source GPT-2, and I hope that OpenAI reverses their decision. What do you all think?
Read more about GPT-2 here:
Tweet
Share