Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

docs : Update installation-deployments.md #706

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
177 changes: 133 additions & 44 deletions docs/installation-deployments.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,17 @@ Check out the [API docs](https://portkey.ai/docs/welcome/make-your-first-request
## Local Deployment

1. Do [NPM](#node) or [Bun](#bun) Install
2. Run a [Node.js Server](https://github.com/Portkey-AI/gateway/blob/main/docs/installation-deployments.md#run-a-nodejs-server)
2. Run a [Node.js Server](#nodejs-server)
3. Deploy on [App Stack](#deploy-to-app-stack)
4. Deploy on [Cloudflare Workers](https://github.com/Portkey-AI/gateway/blob/main/docs/installation-deployments.md#deploy-to-cloudflare-workers)
5. Deploy using [Docker](https://github.com/Portkey-AI/gateway/blob/main/docs/installation-deployments.md#deploy-using-docker)
6. Deploy using [Docker Compose](https://github.com/Portkey-AI/gateway/blob/main/docs/installation-deployments.md#deploy-using-docker-compose)
4. Deploy on [Cloudflare Workers](#cloudflare-workers)
5. Deploy using [Docker](#docker)
6. Deploy using [Docker Compose](#docker-compose)
7. Deploy on [Replit](#replit)
8. Deploy on [Zeabur](https://github.com/Portkey-AI/gateway/blob/main/docs/installation-deployments.md#deploy-to-zeabur)
8. Deploy on [Zeabur](#zeabur)
9. Deploy with [Supabase Functions](#supabase-functions)
10. Deploy using [Fastly](#fastly)

---

### Node

Expand All @@ -39,7 +43,35 @@ $ bunx @portkey-ai/gateway

<br>

# Deploy to App Stack
---

### NodeJS Server

1. Clone the Repository

```sh
git clone https://github.com/portkey-ai/gateway
```

2. Install the NPM Dependencies

```sh
cd gateway
npm i
npm run build
```

3. Run the Server

```sh
node build/start-server.js
```

<br>

---

### Deploy to App Stack
F5 Distributed Cloud
1. [Create an App Stack Site](https://docs.cloud.f5.com/docs/how-to/site-management/create-voltstack-site)

Expand Down Expand Up @@ -94,7 +126,8 @@ curl --request POST \
--data '{"metadata": {"name": "$DISTRIBUTED_CLOUD_SERVICE_NAME","namespace": "$DISTRIBUTED_CLOUD_NAMESPACE","labels": {},"annotations": {},"description": "","disable": false},"spec": {"origin_servers": [{"k8s_service": {"service_name": "$DISTRIBUTED_CLOUD_SERVICE_NAME.$DISTRIBUTED_CLOUD_APP_STACK_NAMESPACE","site_locator": {"site": {"tenant": "$DISTRIBUTED_CLOUD_TENANT_ID","namespace": "system","name": "$DISTRIBUTED_CLOUD_APP_STACK_SITE"}},"inside_network": {}},"labels": {}}],"no_tls": {},"port": 8787,"same_as_endpoint_port": {},"healthcheck": [],"loadbalancer_algorithm": "LB_OVERRIDE","endpoint_selection": "LOCAL_PREFERRED","advanced_options": null}}'
```
or [use the UI](https://docs.cloud.f5.com/docs/how-to/app-networking/origin-pools)
5. Create an HTTP Load Balancer, including header injection of Portkey provider and credentials

6. Create an HTTP Load Balancer, including header injection of Portkey provider and credentials
```shell
curl --request POST \
--url https://$DISTRIBUTED_CLOUD_TENANT.console.ves.volterra.io/api/config/namespaces/$DISTRIBUTED_CLOUD_NAMESPACE/http_loadbalancers \
Expand All @@ -103,7 +136,8 @@ curl --request POST \
--data '{"metadata": {"name": "$DISTRIBUTED_CLOUD_SERVICE_NAME","namespace": "$DISTRIBUTED_CLOUD_NAMESPACE","labels": {},"annotations": {},"description": "","disable": false},"spec": {"domains": ["$PORTKEY_GATEWAY_FQDN"],"https_auto_cert": {"http_redirect": true,"add_hsts": false,"tls_config": {"default_security": {}},"no_mtls": {},"default_header": {},"enable_path_normalize": {},"port": 443,"non_default_loadbalancer": {},"header_transformation_type": {"default_header_transformation": {}},"connection_idle_timeout": 120000,"http_protocol_options": {"http_protocol_enable_v1_v2": {}}},"advertise_on_public_default_vip": {},"default_route_pools": [{"pool": {"tenant": "$DISTRIBUTED_CLOUD_TENANT_ID","namespace": "$DISTRIBUTED_CLOUD_NAMESPACE","name": "$DISTRIBUTED_CLOUD_SERVICE_NAME"},"weight": 1,"priority": 1,"endpoint_subsets": {}}],"origin_server_subset_rule_list": null,"routes": [],"cors_policy": null,"disable_waf": {},"add_location": true,"no_challenge": {},"more_option": {"request_headers_to_add": [{"name": "x-portkey-provider","value": "$PORTKEY_PROVIDER","append": false},{"name": "Authorization","value": "Bearer $PORTKEY_PROVIDER_AUTH_TOKEN","append": false}],"request_headers_to_remove": [],"response_headers_to_add": [],"response_headers_to_remove": [],"max_request_header_size": 60,"buffer_policy": null,"compression_params": null,"custom_errors": {},"javascript_info": null,"jwt": [],"idle_timeout": 30000,"disable_default_error_pages": false,"cookies_to_modify": []},"user_id_client_ip": {},"disable_rate_limit": {},"malicious_user_mitigation": null,"waf_exclusion_rules": [],"data_guard_rules": [],"blocked_clients": [],"trusted_clients": [],"api_protection_rules": null,"ddos_mitigation_rules": [],"service_policies_from_namespace": {},"round_robin": {},"disable_trust_client_ip_headers": {},"disable_ddos_detection": {},"disable_malicious_user_detection": {},"disable_api_discovery": {},"disable_bot_defense": {},"disable_api_definition": {},"disable_ip_reputation": {},"disable_client_side_defense": {},"csrf_policy": null,"graphql_rules": [],"protected_cookies": [],"host_name": "","dns_info": [],"internet_vip_info": [],"system_default_timeouts": {},"jwt_validation": null,"disable_threat_intelligence": {},"l7_ddos_action_default": {},}}'
```
or [use the UI](https://docs.cloud.f5.com/docs/how-to/app-networking/http-load-balancer)
6. Test the service

7. Test the service
```shell
curl --request POST \
--url https://$PORTKEY_GATEWAY_FQDN/v1/chat/completions \
Expand Down Expand Up @@ -137,6 +171,7 @@ in addition to the response headers, you should get a response body like
}
```

---

### Cloudflare Workers

Expand All @@ -159,33 +194,10 @@ npm install
npm run deploy
```

For more details, refer to [Cloudflare Workers official](https://developers.cloudflare.com/workers/).
<br>

### NodeJS Server

1. Clone the Repository

```sh
git clone https://github.com/portkey-ai/gateway
```

2. Install the NPM Dependencies

```sh
cd gateway
npm i
npm run build
```

<br>

3. Run the Server

```sh
node build/start-server.js
```

<br>
---

### Docker

Expand All @@ -199,6 +211,8 @@ For more information on the Docker image, check [here](https://hub.docker.com/r/

<br>

---

### Docker Compose

1. Download Compose File from the Repository:
Expand All @@ -214,50 +228,125 @@ docker compose up -d
```
> The service is now running and listening on port 8787

For more details, refer to [Docker Compose official](https://docs.docker.com/compose/).
<br>

---

### Replit

[![Deploy on Replit](https://replit.com/badge?caption=Deploy%20on%20Replit)](https://replit.com/@portkey/AI-Gateway?v=1)

<br>
---

### Zeabur

[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/templates/RU38E3)

<br>
---

### Vercel
### Supabase Functions

Docs to be written, please help!
1. Clone the Repository:

```sh
git clone https://github.com/portkey-ai/gateway
```

2. Set up a Supabase Account and create a new project (in browser).
3. Install the Supabase CLI, login using `supabase login
` and Initialize, follow [guides](https://supabase.com/docs/guides/local-development)
4. Write Your Supabase Function in:
```sh
supabase/functions/your-function-file/index.ts
```
5. Deploy the Function (for windows add `npx` in start):
```sh
supabase functions deploy your-function-file
```
6. Now, its ready to Test, Monitor and Manage Functions (Using url from supabase dashboard):
- Similar to `https://your-project-id.supabase.co/functions/v1/your-function-file`

Refer to [Supabase's official documentation](https://supabase.com/docs) for more details.

<br>

---

### Fastly

Docs to be written, please help!
1. Create / login to your fastly account (in browser).

2. Clone the Gateway Repository:

```sh
git clone https://github.com/portkey-ai/gateway
```

3. Install Fastly’s CLI:

```sh
npm install fastly-cli
```

4. Log in to Fastly:
```sh
fastly login
```

5. Create a New Fastly Service:
```sh
fastly compute init
```

6. Write Your Edge Function Code in `src/main.rs` or `src/index.js`

7. Set up the token locally (token is located on `Account > User(left bottom most) > API Token`)
```sh
fastly profile create
```
Then follow the steps there to add token.

8. Deploy to Fastly
```sh
fastly compute publish
```
9. Now, its ready to Test, Monitor and Manage Functions (Using url from fastly dashboard):
- Similar to `your-new-project.edgecompute.app`

For more details, refer to [Fastly’s official](https://www.fastly.com/products/edge-compute).

<br>

### AWS Lambda
---

### Vercel

Docs to be written, please help!
- Docs to be written, please help!
- Make sure you've deployed and tested the same.

<br>

### Lambda@edge
---

### AWS Lambda

Docs to be written, please help!
- Docs to be written, please help!
- Make sure you've deployed and tested the same.

<br>

### Supabase Functions
---

### Lambda edge

Docs to be written, please help!
- Docs to be written, please help!
- Make sure you've deployed and tested the same.

<br>

---

## Enterprise Deployment
Make your AI app more <ins>reliable</ins> and <ins>forward compatible</ins>, while ensuring complete <ins>data security</ins> and <ins>privacy</ins>.

Expand Down
Loading