Should you use OpenAI's embeddings? Probably not, and here's why. In order to answer this question, we need to consider the language model that will provide the answers and the embeddings model that will select information from the knowledge base. Currently, OpenAI is the leading choice for the language model, with GPT4 and GPT3.5 being the top options. However, when it comes to embeddings, OpenAI's offerings are not clearly superior.

Should you use OpenAI's embeddings? Probably not, and here's why. In order to answer this question, we need to consider the language model that will provide the answers and the embeddings model that will select information from the knowledge base. Currently, OpenAI is the leading choice for the language model, with GPT4 and GPT3.5 being the top options. However, when it comes to embeddings, OpenAI's offerings are not clearly superior.

Benchmarks, such as the one mentioned here, indicate that there are models that perform better than ada-002, OpenAI's embedding model. Particularly, the Instructor models like xl and large have shown impressive results. Of course, benchmarks alone may not be sufficient in determining the right choice for you. Factors like cost, performance, and speed need to be considered to find the optimal compromise.

Another aspect to consider is the trust you have in OpenAI to maintain the availability of the model in the future. What if ada-002 is discontinued after you have embedded millions of documents? Or what if your usage increases exponentially, and you find yourself embedding millions of queries per day at a potentially high cost charged by OpenAI for the API?

To address these concerns, it is recommended to randomize queries using both sets of embeddings to see if any noticeable difference can be observed. Surprisingly, in many cases, there may not be a significant distinguishable distinction. Based on this observation, the following procedure is suggested:

1. Try the lightest embedding model initially. See if it meets your requirements and delivers satisfactory results.
2. If the lightest model falls short, consider using a more powerful embedding model and conduct a blind comparison to assess its performance.
3. Only if you are already utilizing a relatively large model like Instructor XL, then try a blind test against OpenAI's ada-002. If you find that OpenAI's model is indeed better suited for your application, then opt for it.

By following this procedure, you can ensure that you explore different options and make an informed decision based on your specific needs and preferences.

In the realm of GPT-4, there is an important issue to be aware of – the prompt injection attack. OpenAI has acknowledged that this type of attack is one of the most effective methods of "breaking" the model. In their GPT-4 System Card published on March 23, they highlight the significance of this vulnerability.

This prompt injection attack involves manipulating the system message to influence the behavior and output of the model. It is a form of adversarial attack that can potentially compromise the robustness and integrity of the GPT-4 model. OpenAI's acknowledgment of this vulnerability emphasizes the need for further research and development to enhance the security and resilience of future language models.

In conclusion, the decision to use OpenAI's embeddings should be made after careful consideration of various factors such as performance, cost, and trust in the availability of the model. Conducting blind tests and exploring different options can help identify the most suitable embedding model for your specific needs. Additionally, staying aware of vulnerabilities like the prompt injection attack can contribute to a more robust and secure use of language models.

Actionable Advice:

1. Experiment with different embedding models: Don't settle for the first option you come across. Explore various models and conduct tests to find the one that best aligns with your requirements.
2. Consider long-term stability: Evaluate the trustworthiness and commitment of the provider in maintaining the availability of the chosen model. Ensure that you won't face unexpected discontinuation or exorbitant costs in the future.
3. Stay informed about security vulnerabilities: Keep yourself updated on potential attacks or vulnerabilities associated with the language models you use. This knowledge will help you take necessary precautions and mitigate risks effectively.