Unfortunately, LLMs trained by next word prediction are difficult to use, often produce factually inaccurate or offensive output, and can be used in harmful applications.

Reinforcement Learning from Human Feedback (RHLF) has been shown to make models considerably more aligned and easier to use. This technique has been used by OpenAI, DeepMind, and Anthropic to produce LLMs that follow instructions or act as helpful assistant.

gatekept models limit their value to academics, hobbyists, and industry. We see a future where RLHF-tuned models will be applied and adapted to every domain and task unlocking huge amounts of real world value.

Carper AI is partnering with Humanloop and Scale to collect and apply the human feedback data that will be used to improve the underlying language model that Carper trains. Humanloop are expert in adapting LLMs from human feedback and Scale are leaders in data annotation. Hugging Face will host the final trained model and make it generally accessible.