Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat: update README.md and Makefile #3

Merged
merged 1 commit into from
Mar 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,22 @@ tg-apply:
tg-destroy:
yes | terragrunt run-all destroy --terragrunt-working-dir ./terragrunt/ai/server

pre-commit:
pre-commit run -a -v

ssh:
MY_IP=$(shell terragrunt output --terragrunt-working-dir ./terragrunt/ai/server ipv4 | xargs); \
ssh-keygen -R "$$MY_IP"; ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@$$MY_IP

tg-init:
yes | terragrunt run-all init -reconfigure --terragrunt-working-dir ./terragrunt/ai/server

tg-apply:
yes | terragrunt run-all apply --terragrunt-working-dir ./terragrunt/ai/server

tg-destroy:
yes | terragrunt run-all destroy --terragrunt-working-dir ./terragrunt/ai/server

ssh:
MY_IP=$(shell terragrunt output --terragrunt-working-dir ./terragrunt/ai/server ipv4 | xargs); \
ssh-keygen -R "$$MY_IP"; ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@$$MY_IP
53 changes: 53 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,55 @@
# private-ai
private AI runnning on Hetzner Infra

## prequisites
- [Hetzner](https://hetzner.cloud/?ref=UTExFIi8vytX) account
- asdf
- terraform

## setup

Install dependencies
```bash
make dependencies
```
note: script [dependencies.sh](scripts%2Fdependencies.sh) was originally written for macos, you might need to adjust it for your system

Create a new project in the Hetzner Cloud Console and create an API token.
Maybe you'll need new project: https://console.hetzner.cloud/projects, and in project go to Security->API Tokens and create new token.

Create new .env.yaml file used by terragrunt.
```bash
cp .env.yaml.example .env.yaml
```
and change values of hetzner token and your pub ssh key.
```yaml
secrets:
hetzner: "token"
pub_ssh_key: "ssh key token"
```

1. Then you can run terragrunt init
```bash
make tg-init
```
2. and Terragrunt apply
```bash
make tg-apply
```

This should create a server at hetzner cloud and install all necessary software on it.

After that you can ssh to the server and run ollama run llama2-uncensored model, which is already pulled.
```bash
make ssh
# on server
ollama run llama2-uncensored
```

## BE CAREFUL
This is just a simple example of how to run private AI on Hetzner Cloud. It's not production ready and should be used only for educational purposes.

Also don't forget destroy your infrastructure after you finish your work, because server which we using is not free and you will be charged for it. CPX51 which we are using cost 65USD per month, better to use it temporarily and destroy it after you finish your work.
```bash
make tg-destroy
```
Loading