RAG – Vercel AI SDK, Next.js, LangChain, Ollama Chat Example

This example shows how to use the Vercel AI SDK with Next.js, LangChain, and Ollama to create a ChatGPT-like AI-powered streaming chat bot.

Deploy your own

NOTE: Ollama needs to be deployed separately.

Deploy the example using Vercel:

How to use

Execute create-next-app with npm, Yarn, or pnpm to bootstrap the example:

npx create-next-app --example https://github.com/brunnolou/next-langchain-rag-ollama

yarn create next-app --example https://github.com/brunnolou/next-langchain-rag-ollama

pnpm create next-app --example https://github.com/brunnolou/next-langchain-rag-ollama

To run the example locally you need to:

Run Ollama

1. Download Ollama and install it locally.
2. run ollama run mistral to download and install the model locally (Requires 4.1GB and 8GB of RAM)
3. Open http://localhost:11434 to check if Ollama is running.

Setup Qdrant vector database

1. Run docker pull qdrant/qdrant and docker run -p 6333:6333 qdrant/qdrant
2. Open the console dashboard localhost:6333/dashboard#/console.
3. Create a new collection running this:

PUT collections/swiss-or
{
  "vectors": {
    "size": 384,
    "distance": "Cosine"
  }
}

4. Download the snapshot file
5. Unzip the file using the terminal (⚠️ not with Finder on Mac ⚠️) with unzip ...
6. Upload the file using the following command. Adapt the fields accordingly and run it from the same directory, as where your snapshot lies.

curl -X POST 'http://localhost:6333/collections/swiss-or/snapshots/upload' \
    -H 'Content-Type:multipart/form-data' \
    -F 'snapshot=@swiss-code-of-obligations-articles-gte-small-2023-10-18-12-13-25.snapshot'

Run the app

1. yarn install to install the required dependencies.
2. Copy the file .env.local.example in the project and rename it to .env. Verify if all environment variables are correct.
3. yarn dev to launch the development server.
4. Go to http://localhost:3000 and try out the app.

Learn More

To learn more about LangChain, OpenAI, Next.js, and the Vercel AI SDK take a look at the following resources:

• Vercel AI SDK docs - learn mode about the Vercel AI SDK
• LangChain Documentation - learn about LangChain
• Ollama - learn about Ollama features, models, and API.
• Next.js Documentation - learn about Next.js features and API.
• Qdrant Documentation - learn about Next.js features and API.