How to Connect Hugging Face Models to LangChain

TL;DR

Install the langchain-huggingface partner package and supporting libraries, create a Hugging Face read token, and expose it through Google Colab secrets or a local environment variable. You can then use HuggingFaceEndpoint to call hosted models through APIs, avoiding the local resource demands of downloading and running models directly.

Transcript

hello guys so in this video and in the upcoming series of video we are going to discuss how we can build generative AI application using hugging face open source models along with Lang chain now if you don't know about hugging face it is an amazing platform it has various stateof art models for machine learning deep learning natural language proces... Read More

Key Insights

• The langchain-huggingface package is a partner integration jointly maintained by Hugging Face and LangChain. It is designed to make Hugging Face libraries and models easier to use within LangChain while keeping the integration aligned with current Hugging Face development.
• Hugging Face Hub contains repositories for models supporting tasks such as text generation, question answering, document question answering, and image-to-text processing. A model can be selected from the Hub and referenced in code through its repository identifier.
• The required setup includes langchain-huggingface, huggingface-hub, transformers, accelerate, bitsandbytes, and LangChain. Transformers supports pipeline-based model usage, while the other installed packages support model access and the development of LangChain applications.
• Google Colab is presented as a feasible environment for following the tutorial. Local Jupyter notebooks and workstations can also execute the code, but downloading and running models locally may require a good amount of RAM, storage, and other computing resources.
• A Hugging Face access token is required to connect code with models available through Hugging Face Hub. The tutorial creates a new token with read functionality through the access-token section of the Hugging Face account settings.
• Google Colab secrets provide a way to store the Hugging Face token under a named key. Python code can retrieve that value with google.colab.userdata.get, while local development can retrieve a stored environment variable by using os.getenv.
• HuggingFaceEndpoint accesses Hugging Face models through hosted APIs instead of downloading the complete model into the notebook environment. The endpoint can use a repository identifier, and the serverless API is especially beneficial for Pro or Enterprise Hub accounts.
• Regular Hugging Face users can also access hosted models through API requests when their Hugging Face token is connected to the execution environment. The imported HuggingFaceEndpoint class comes from langchain_huggingface, making the API integration more seamless than the earlier multi-library approach.

Questions & Answers

Q: How do you start using Hugging Face models with LangChain?

Install langchain-huggingface together with huggingface-hub, transformers, accelerate, bitsandbytes, and LangChain. Next, create a Hugging Face account and generate a token with read access. Store that token as a Google Colab secret or a local environment variable, then import HuggingFaceEndpoint from langchain_huggingface to connect to a selected hosted model.

Q: What is the langchain-huggingface partner package?

The langchain-huggingface package is an integration announced by Hugging Face and LangChain and jointly maintained by both organizations. It connects Hugging Face libraries and models with the LangChain ecosystem. Its purpose is to make model calls more seamless and keep the integration current as Hugging Face development continues.

Q: Which libraries are installed for the Hugging Face setup?

The setup installs langchain-huggingface, huggingface-hub, transformers, accelerate, bitsandbytes, and LangChain. The partner package supplies the direct LangChain integration, Hugging Face Hub provides access to model repositories, and Transformers supports pipeline-based model calls. LangChain is included for building the generative AI applications demonstrated throughout the series.

Q: How do you create a Hugging Face access token?

Create a Hugging Face account, open the settings page, and navigate to the access-token section. Select the option to create a new token, give it a name, assign read functionality, and generate it. Copy the resulting token so it can be stored in Google Colab secrets or an environment variable for authenticated model access.

Q: How do you store a Hugging Face token in Google Colab?

Open the secrets area in Google Colab, add a new secret, enter the chosen token key name, and paste the Hugging Face token as its value. Keep the secret enabled for the notebook. In Python, import userdata from google.colab and call userdata.get with the secret name to retrieve the stored value.

Q: How can a Hugging Face token be accessed in local code?

When working locally, such as in VS Code or a Jupyter notebook, store the Hugging Face token in an environment variable. Import the Python os module and retrieve the value with os.getenv using the variable's key. The retrieved token can then be assigned to the environment entry expected by the Hugging Face integration.

Q: What is HuggingFaceEndpoint used for in LangChain?

HuggingFaceEndpoint is imported from the langchain_huggingface package and is used to access hosted Hugging Face models through APIs. A model can be identified with its repository ID, allowing code to call it through an endpoint. This method avoids downloading the entire model into the local or Google Colab environment.

Q: Why use a hosted Hugging Face endpoint instead of a local model?

A hosted endpoint lets the notebook access a Hugging Face model through an API rather than downloading the model into local RAM. Running a downloaded model may require a good amount of memory, storage, and computing resources. Regular users can still make a fair amount of endpoint requests by connecting their Hugging Face token.

Summary & Key Takeaways

• The tutorial introduces the jointly maintained langchain-huggingface partner package for building generative AI applications with Hugging Face models and LangChain. The series focuses mainly on free, open-source models and demonstrates each part of the code line by line, using Google Colab as the accessible execution environment.

• Initial setup requires installing langchain-huggingface, huggingface-hub, transformers, accelerate, bitsandbytes, and LangChain. A Hugging Face account and read access token are also needed. In Google Colab, the token can be saved as an enabled secret and retrieved with userdata.get instead of being inserted directly throughout the notebook.

• HuggingFaceEndpoint provides a way to access hosted Hugging Face models through APIs by specifying a model repository identifier. This approach contrasts with downloading a model into local RAM, which can require substantial computing resources. Regular users can make a fair amount of requests by connecting their Hugging Face token.