A Guide to Curation in Community: Aligning Language Models to Follow Instructions

A Guide to Curation in Community: Aligning Language Models to Follow Instructions

In today's digital age, the abundance of information can be overwhelming. As a community builder, it is your responsibility to curate content and provide a valuable resource for your members. But community curation is not just about creating a repository of hand-picked information; it is an exercise that creates immense value for both you and your members.

For your members, the value of community curation is clear. They have access to trusted and vetted information, saving them time and effort in sifting through the vast sea of content. A curated source becomes a go-to platform for quick and reliable information. However, the true value of curation lies in the act itself.

As a community builder, when you curate content, you dive deep into the community you serve. You process and understand the information shared by your members. In this process, you inevitably learn and discover new ideas. These ideas can then be shared with your community, fostering a cycle of continuous learning and growth.

But how do you decide what content to curate? Before saving anything, ask yourself if it is worth saving. Is it valuable? Is it relevant to your community's interests and goals? Curating everything may dilute the value of your curation efforts. Remember, if everything is special, then nothing is special. Be selective and intentional in what you choose to curate.

Curation is not just about creating a resource for your members; it is about connecting the dots. A repository of information can be helpful for anyone in a business or community, but the true power of curation lies in your ability to make connections. As a community builder, you have the unique opportunity to see patterns and trends within your community. By curating content that aligns with these patterns, you become a valuable resource not only for your members but potentially for every employee in your company.

Now, let's shift our focus to aligning language models to follow instructions. InstructGPT models have shown significant improvements in following instructions compared to GPT-3 models. While GPT-3 is trained to predict the next word based on a large dataset of internet text, InstructGPT models are specifically designed to perform language tasks as instructed by the user. They are aligned with their users' needs and preferences.

To make language models safer, more helpful, and more aligned, reinforcement learning from human feedback (RLHF) is used. By fine-tuning the models on a small curated dataset of human demonstrations, harmful outputs can be reduced. In fact, our labelers prefer outputs from the InstructGPT model over the GPT-3 model, despite the former having significantly fewer parameters. This showcases the power of curation in improving the performance and safety of language models.

However, it is important to note that InstructGPT models are still not fully aligned or fully safe. They can still generate toxic or biased outputs, make up facts, and even generate explicit content without explicit prompting. Refusing certain instructions reliably is a crucial research problem that needs to be addressed to mitigate the risk of misuse.

Additionally, InstructGPT models are currently biased towards the cultural values of English-speaking people. To overcome this limitation, research is being conducted to understand the differences and disagreements between labelers' preferences. This will allow models to be conditioned on the values of more specific populations, ensuring inclusivity and fairness.

In conclusion, community curation and aligning language models to follow instructions are two distinct yet interconnected concepts. As a community builder, curating content not only benefits your members but also provides valuable insights and knowledge for yourself. By connecting the dots and curating with intention, you become a trusted resource for your community and potentially for your entire organization.

To make the most out of community curation and aligning language models, here are three actionable pieces of advice:

1. Be selective in your curation efforts. Curate content that is valuable and relevant to your community's interests and goals. This will ensure that your curated resource remains special and impactful.

2. Continuously learn and share new ideas. As you curate content and discover new insights, pass on this information to your community. Foster a culture of continuous learning and growth within your community.

3. Advocate for safe and aligned language models. Support initiatives that focus on aligning language models with user needs and preferences. Encourage the development of models that refuse certain instructions and prioritize safety and inclusivity.

By following these three pieces of advice, you can enhance the value of your community curation efforts and contribute to the development of safer and more aligned language models. Remember, community curation is not just about creating a resource; it is an exercise in learning, understanding, and connecting the dots. Embrace the power of curation and make a positive impact on your community and beyond.