diff --git a/Makefile b/Makefile index 04413eb..b2654ca 100644 --- a/Makefile +++ b/Makefile @@ -32,6 +32,22 @@ tg-apply: tg-destroy: yes | terragrunt run-all destroy --terragrunt-working-dir ./terragrunt/ai/server +pre-commit: + pre-commit run -a -v + +ssh: + MY_IP=$(shell terragrunt output --terragrunt-working-dir ./terragrunt/ai/server ipv4 | xargs); \ + ssh-keygen -R "$$MY_IP"; ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@$$MY_IP + +tg-init: + yes | terragrunt run-all init -reconfigure --terragrunt-working-dir ./terragrunt/ai/server + +tg-apply: + yes | terragrunt run-all apply --terragrunt-working-dir ./terragrunt/ai/server + +tg-destroy: + yes | terragrunt run-all destroy --terragrunt-working-dir ./terragrunt/ai/server + ssh: MY_IP=$(shell terragrunt output --terragrunt-working-dir ./terragrunt/ai/server ipv4 | xargs); \ ssh-keygen -R "$$MY_IP"; ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@$$MY_IP diff --git a/README.md b/README.md index 32d1f06..fea357d 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,55 @@ # private-ai private AI runnning on Hetzner Infra + +## prequisites +- [Hetzner](https://hetzner.cloud/?ref=UTExFIi8vytX) account +- asdf +- terraform + +## setup + +Install dependencies +```bash +make dependencies +``` +note: script [dependencies.sh](scripts%2Fdependencies.sh) was originally written for macos, you might need to adjust it for your system + +Create a new project in the Hetzner Cloud Console and create an API token. +Maybe you'll need new project: https://console.hetzner.cloud/projects, and in project go to Security->API Tokens and create new token. + +Create new .env.yaml file used by terragrunt. +```bash +cp .env.yaml.example .env.yaml +``` +and change values of hetzner token and your pub ssh key. +```yaml +secrets: + hetzner: "token" + pub_ssh_key: "ssh key token" +``` + +1. Then you can run terragrunt init +```bash +make tg-init +``` +2. and Terragrunt apply +```bash +make tg-apply +``` + +This should create a server at hetzner cloud and install all necessary software on it. + +After that you can ssh to the server and run ollama run llama2-uncensored model, which is already pulled. +```bash +make ssh +# on server +ollama run llama2-uncensored +``` + +## BE CAREFUL +This is just a simple example of how to run private AI on Hetzner Cloud. It's not production ready and should be used only for educational purposes. + +Also don't forget destroy your infrastructure after you finish your work, because server which we using is not free and you will be charged for it. CPX51 which we are using cost 65USD per month, better to use it temporarily and destroy it after you finish your work. +```bash +make tg-destroy +```