I signed up for the OpenAI API Beta when GPT-3 was all the rage last year. Having finally gotten out of the waitlist a month back, I promptly jumped onto the OpenAI testing bandwagon. I wanted to take a stab at a concept that always intrigued me: use NLP to completely organically generate something in the realm of art, like a poem or a lyric.
As I dived into the OpenAI API documentation, I quickly realized the use case of Completions is the most relevant to me since I want to let users provide a prompt, and let the API return a poem. The OpenAI Playground which you need an account to access has been hugely helpful to me in trying out various models (engines) and tuning parameters. Since my desired output is flexible, I used parameters mostly based on gut feel of how the frequently the output returned makes sense. The following parameters are up for tuning:
- Response length/max tokens
- Temperature [0 to 1]: Sampling temperature, or how much risk the model takes in putting the response together.
- Top_p [0 to 1]: Alternative solution to the sampling problem besides temperature sampling. There’s a lot more math in the linked literature, but basically results with tokens below top_p probability mass are not considered.
- Presence penalty [>0, suggested to ≤ 1]: Penalty on new tokens based on if they have appeared in the response so far.
- Frequency penalty [>0, suggested to ≤ 1]: Penalty on new tokens based on the frequency of them appearing in the response so far.
- Stop sequence
In addition, there are a few other API parameters which are less relevant to the content of the response.
Since I want my poetry writing app to be pretty creative, I set the above parameters to give a high degree of chaos and less repetition: 0.6–0.7 range for temperature, 0.3 for presence penalty, 0.5 for frequency penalty. The stop sequence is a bit of a double-edged sword. On the one hand, I want to set common stop tokens (full stop, exclamations) as stop sequences to encourage the response to terminate appropriately. On the other hand, poems have these punctuations without terminating, so what ends up happening is that the poem terminates prematurely. After experimenting, I left the stop sequence as a placeholder ‘###’.
Other than tuning parameters, it is also important to choose the right engine for the job. Based on Playground tests, davinci and davinci-instruct-beta return the best results for poem generation. The other engines are supposed to be better at specific tasks such as classification, translation or sentiment analysis. In fact, davinci-instruct-beta is slightly better for my task since it is trained to understand “shorter and more naturally phrased prompts”, which is what my poetry prompt would be. However, davinci-instruct-beta is currently not open to API endpoints, so we have to do with davinci for now.
Choosing the right prompt is also an important part of getting a good response since that is the only input you give to the model. I settled on
Poem about <topic>:<\n>Either “poem” or “poetry” works, though “poem” suggests a bit more about the form of the output. The addition of :<\n> is intentional, to signal clearly to the model that the output should not be a sentence completion to “Poem about <topic>” but a separate textual entity — that begins on a new line.
Having chosen these basic inputs, I set about putting an app together. I wanted simple frontend to give visual context to calling the endpoint and displaying the results. I decided to use the React framework because it has an easy bootstrap process that gives a pretty nicely templated home page. All I had to do was to add a form component that takes in a prompt, calls the endpoint, and displays the result.
You can check out my code and some sample generated poetry here: https://github.com/qbiqing/openai-poems. By the way, to publish an OpenAI API application one has to submit a review request to the OpenAI team, which is why I will leave mine non-hosted for the time being.
In the most recent commit, I added extra fields to the form, so users can specify their own desired max length and temperature (I call it chaos). Here are some small poems I am quite happy with:
I have some improvements in mind the progress of which I will probably update in the next post:
- Allow users to choose different types of poems to generate, eg. sonnet, haiku. This probably involves demonstrating the desired format of the poem through the prompt.
- Allow users to customize the number of lines of the poem. Need to experiment with achieving this while still finishing the poem naturally.
- I wonder if I can give OpenAI some sort of input to indicate a song and have it fill in the lyrics?
Ending this blog post with a gentle poem by OpenAI:
The heart is not a stone,
Nor is it dead,
It feels, and knows its own grief.
