How Can You Fine-Tune GPT-3 to Write a Coherent Novel? (Part 1)

TL;DR

To fine-tune GPT-3 for coherent fiction, David Shapiro recommends using an empty prompt with a whole story or large section as the completion, or training on consecutive paragraph pairs. He also tests a lore book, previous-story context, higher temperature, and a modest frequency penalty in his Auto Muse 2 experiment. Read on for eight grounded answers about his proposed workflow and early findings.

Transcript

hey everybody david shapiro here um so there was a post on the open ai forum that was asking about well here let me just show you um what was it the guy was asking about where was it um he was asking how to fine tune from scratch uh oh it was this one okay okay so let me show you guys this so this guy asked i'm using short stories that i wrote to b... Read More

Key Insights

• ✍️ Fine-tuning GPT-3 for fiction writing remains a challenging task, and achieving coherent fiction is elusive.
• 😃 The document example of fine-tuning, where the whole story or a big chunk of it is used as the completion, can help fine-tune the model to match the writer's style and tone.
• 📔 The lore book mechanism, as used by AI Dungeon, can assist in maintaining consistency and referencing major story details in each completion.
• ✍️ Experimenting with different prompts and completion formats can provide valuable insights into the capabilities and limitations of GPT-3 for fiction writing.
• 😒 The auto muse 2 experiment aims to generate novel-length fiction by combining story premises, outlines, and summaries to generate coherent and engaging stories.
• ✍️ The generation of fan fiction and the ability to write in different styles demonstrate the flexibility and adaptability of GPT-3 in fiction writing.

Questions & Answers

Q: How can GPT-3 be fine-tuned to generate coherent fiction?

David Shapiro recommends following the document fine-tuning example: leave the prompt empty and place the whole story or a large section in the completion. He says this requires more samples but can tune the model to the writer’s style and tone.

Q: What alternative fine-tuning format does David Shapiro suggest for fiction?

He suggests using one paragraph as the prompt and the subsequent paragraph as the completion. This trains GPT-3 to follow an input paragraph with the next part of the story.

Q: What is the lore book mechanism described in the video?

Shapiro says AI Dungeon apparently uses a lore book so major story details can be referenced during every completion. He proposes including a lore book section and previous paragraphs in the input, with the next paragraphs serving as the completion.

Q: What is the goal of the Auto Muse 2 experiment?

Auto Muse 2 is an experiment to generate novel-length fiction from a single story premise. Shapiro describes the work as rapid prototyping and begins by recycling code and testing prompts.

Q: What source material does the Auto Muse 2 experiment begin with?

Shapiro starts with a folder containing 200 story premises generated for an earlier project. He inserts a premise into a prompt asking GPT-3 to write a novel or story from it.

Q: Can GPT-3 imitate a requested writing style?

Shapiro says his earlier tests showed that GPT-3 changes its writing when prompted to write like a Victorian gentleman, Shakespeare, or a chav. In this experiment, he asks it to write like Frank Herbert while acknowledging ethical implications.

Q: Why does David Shapiro raise the temperature for fiction generation?

He says fiction generally benefits from a higher temperature because it makes the output more creative and less deterministic. He increases it after GPT-3 appears to copy the supplied premise nearly verbatim.

Q: How does frequency penalty affect GPT-3 fiction generation in Shapiro’s tests?

Shapiro says raising the frequency penalty slightly can make the output more creative. During his movie script generator work, combining a higher frequency penalty with a higher temperature reduced repeated story patterns somewhat.

Summary & Key Takeaways

• A user on the OpenAI forum asks for tips on fine-tuning GPT-3 for generating fiction.

• David Shapiro recommends using the document example of fine-tuning or the lore book mechanism used by AI Dungeon.

• Shapiro conducts experiments to generate fiction, using different story premises and writing styles.