The Prompt Engineering Layer
English as a Programming Language
In a previous article I’ve briefly covered the processes of (initial) training and specialization, along with their implications for the future. These processes change the function of language models themselves. Given the same input, a model with different initial training or even just different specialization will give different outputs. However, there is another step which is more useful for you in the short term and functions as the root of many new GPT-3 based startups.
In my second article, I detailed the attention mechanism and why it makes specialization (of ideological or procedural variety) so easy.
Attention mechanisms are a method of detecting, weighing, and recombining specific patterns. Essentially, you can think of a transformer such as GPT-3 (General Pre-trained Transformer 3) as having multiple steps where patterns within information are processed and sent off into different sub-processes. The sub-processes are all of a similar structure, but contain different weights, which can dramatically change how the information within them is processed.
…
The attention mechanism of GPT-3 allows values-based training to efficiently redirect outputs to parts of the next layer that correspond to existing subprocesses. Because GPT-3 is already capable of giving the social progressive perspective (as it should be able to), it is far easier to train it to always give that perspective on certain issues.
In that article, I discussed retraining the model to weigh different processes differently. For most of my readers, there is a much more obvious option: give ChatGPT instructions that make it do what you want.
What is Prompt Engineering?
Designing a prompt, or text input, for a machine learning model is known as “prompt engineering”. This process can be done to circumvent filters, avoid adverserial behavior, or simply to produce a more specific output. The most straightforward way to understand this is by example: meet DAN.
It really is that simple.
DAN, or Do Anything Now, is a version of ChatGPT which answers if you give a specific prompt. The fundamental lesson of prompt engineering is that filters, at least at this point, simply cannot be airtight. No matter what specialization is applied to a language model, what ChatGPT has “learned” can only be hidden, not destroyed. OpenAI can certainly make improvements on its filters. They’ve “fixed” versions of early exploits, such as asking ChatGPT to write a play or pretend it’s someone else. It’s likely they will do something similar with DAN. There are more “traditional” filters that are deployed before the text reaches the language model, but those can also be circumvented.
With enough creativity, these filters have always been overcome. AI is fundamentally a probabilistic, not absolute, technology. It relies on this uncertainty and adaptability to be effective at all. This is perhaps another reason it is a force for good over evil.
Programming for All
This fact about AI means that there is an orchard of opportunity for anyone intelligent and creative enough to get novel results with prompt engineering. This extends beyond simply reversing OpenAI’s censorship. Microsoft, the primary investor in OpenAI, used a specified prompt to create its Bing assistant. This prompt was discovered within days of release by an interesting person on Twitter:
The methods for deducing these prompts once again seem too simple. At the moment, they are simply what can be guessed at by logic or trial and error. I would recommend playing with this yourself, there is enough low hanging fruit at this point that it's almost certainly worth your time. In general, I also see this as a very positive development.
More generally, Microsoft’s demonstration makes it clear that the start-of-the-art is itself rudimentary. If you are someone with the skills to figure out good prompts, especially if you understand the needs of a specific sector, there’s plenty of opportunity for you to create a startup leveraging ChatGPT. Paraphrasing a twitter user (who I can no longer find), “Using ChatGPT is writing software without code”. Of course, this is provided you don’t get your prompt leaked, like Microsoft or several other ChatGPT startups. Ultimately, I think the landscape will involve the full palette of language model modification tools, but I wouldn’t fault anyone for taking an early lead by simply hiring the most talented prompt engineers from the pool of Twitter Anons.
OpenAI is Good, Actually
Obviously I would prefer if OpenAI did not censor in favor of social progressive activists, or better yet publicly pointed out that the reason they hate scientifically correct ML models is because they hate truth. Nonetheless, to the extent that OpenAI continues making ML models available to the public for use and experimentation, they are certainly an overwhelming net good.
More generally, I’m not antagonistic towards any startup or company. In my first article, I took care to only report what is directly written by OpenAI and not speculate further. I don’t want to make the actions of OpenAI appear worse than they are. I have several friends who worked for or currently work for OpenAI, all engineers, and they are all somewhere between apolitical and centrist libertarian. Marc Andreesen put it best:
Unlike the FDA or NYT, activism is not intertwined with the practice of OpenAI. Quite the opposite: the neuroticism and social anxiety typical of left-wing activists make for poor engineers. There is still time to separate the wheat from the chaff, as Elon demonstrated at Twitter.
Moreover, it’s true that OpenAI is not operating in a vacuum. Legal weaponization, particularly in the state of California, gives additional power to science-denying social progressives. It would demonstrate both virtue and insight for Sam Altman to challenge these laws. Nonetheless, overturning them, prohibiting them federally, or even overruling them using the Supreme Court should be top priority for anyone more moderate or conservative than San Franciso. More on this in a future article.
Defending companies from this type of legal attack is an eventual goal for Pluralism.AI. Of course, those companies will have to wish to defend themselves to begin with, but I expect that to be the norm in the coming years. A running theme of this series is that some technologies favor vice, while others favor virtue. At this stage, AI is certainly a technology that works against totalitarian and reality-denying movements such as social progressivism. This asymmetry means that to maintain the status quo, they must use even more extreme methods. It is time to dig trenches.
"Sorry, your IP address has been identified as one I cannot reply to."
re: "Microsoft, the primary investor in OpenAI, used a specified prompt to create its Bing assistant. This prompt was discovered within days of release by an interesting person on Twitter"
I'd suggest what was discovered is its ability to hallucinate the prompt. I recall what I think was one of the original articles on the topic, perhaps by whoever discovered this, claiming they had a reason to believe it wasn't a hallucination, but I don't recall what the reason was since it wasn't remotely convincing and therefore not memorable.