Skip to content

Latest commit

 

History

History
78 lines (55 loc) · 3.54 KB

README.md

File metadata and controls

78 lines (55 loc) · 3.54 KB
Raw

RAG – Vercel AI SDK, Next.js, LangChain, Ollama Chat Example

This example shows how to use the Vercel AI SDK with Next.js, LangChain, and Ollama to create a ChatGPT-like AI-powered streaming chat bot.

Deploy your own

NOTE: Ollama needs to be deployed separately.

Deploy the example using Vercel:

Deploy with Vercel

How to use

Execute create-next-app with npm, Yarn, or pnpm to bootstrap the example:

npx create-next-app --example https://github.com/brunnolou/next-langchain-rag-ollama
yarn create next-app --example https://github.com/brunnolou/next-langchain-rag-ollama
pnpm create next-app --example https://github.com/brunnolou/next-langchain-rag-ollama

To run the example locally you need to:

Run Ollama

  1. Download Ollama and install it locally.
  2. run ollama run mistral to download and install the model locally (Requires 4.1GB and 8GB of RAM)
  3. Open http://localhost:11434 to check if Ollama is running.

Setup Qdrant vector database

  1. Run docker pull qdrant/qdrant and docker run -p 6333:6333 qdrant/qdrant
  2. Open the console dashboard localhost:6333/dashboard#/console.
  3. Create a new collection running this:
PUT collections/swiss-or
{
  "vectors": {
    "size": 384,
    "distance": "Cosine"
  }
}
  1. Download the snapshot file
  2. Unzip the file using the terminal (⚠️ not with Finder on Mac ⚠️) with unzip ...
  3. Upload the file using the following command. Adapt the fields accordingly and run it from the same directory, as where your snapshot lies.
curl -X POST 'http://localhost:6333/collections/swiss-or/snapshots/upload' \
    -H 'Content-Type:multipart/form-data' \
    -F 'snapshot=@swiss-code-of-obligations-articles-gte-small-2023-10-18-12-13-25.snapshot'

Run the app

  1. yarn install to install the required dependencies.
  2. Copy the file .env.local.example in the project and rename it to .env. Verify if all environment variables are correct.
  3. yarn dev to launch the development server.
  4. Go to http://localhost:3000 and try out the app.

Learn More

To learn more about LangChain, OpenAI, Next.js, and the Vercel AI SDK take a look at the following resources: