Skip to content

@deepinfra Deep Infra

Change the repository type filter

All

    Repositories list

    26 repositories

    • TensorRT-LLM

      Public
      TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
      C++
      •
      Apache License 2.0
      •1.1k•0•0•0•Updated Feb 27, 2025Feb 27, 2025

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •5.9k•0•0•0•Updated Feb 26, 2025Feb 26, 2025

    • tensorrtllm_backend

      Public
      The Triton TensorRT-LLM Backend
      Python
      •
      Apache License 2.0
      •114•0•0•0•Updated Feb 25, 2025Feb 25, 2025

    • flash-attention

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •1.5k•0•0•0•Updated Feb 20, 2025Feb 20, 2025

    • cutlass

      Public
      CUDA Templates for Linear Algebra Subroutines
      C++
      •
      Other
      •1.1k•0•0•0•Updated Feb 20, 2025Feb 20, 2025

    • Zonos

      Public
      Python
      •
      Apache License 2.0
      •557•0•0•0•Updated Feb 12, 2025Feb 12, 2025

    • sglang

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •1.1k•0•0•0•Updated Feb 4, 2025Feb 4, 2025

    • Pyramid-Flow

      Public
      Code of Pyramidal Flow Matching for Efficient Video Generative Modeling
      Python
      •
      MIT License
      •274•0•0•0•Updated Oct 21, 2024Oct 21, 2024

    • llama-stack

      Public
      Model components of the Llama Stack APIs
      Python
      •
      MIT License
      •895•0•0•0•Updated Oct 10, 2024Oct 10, 2024

    • ngx-http-auth-jwt-module

      Public
      Secure your NGINX locations with JWT
      Shell
      •
      MIT License
      •125•0•0•0•Updated Jun 17, 2024Jun 17, 2024

    • deepctl

      Public
      Command line tool for Deep Infra cloud ML inference service
      Rust
      •
      Apache License 2.0
      •1•29•1•0•Updated Jun 10, 2024Jun 10, 2024

    • langchainjs

      Public
      🦜🔗 Build context-aware reasoning applications 🦜🔗
      TypeScript
      •
      MIT License
      •2.4k•0•0•0•Updated May 31, 2024May 31, 2024

    • deepinfra-node

      Public
      Official TypeScript wrapper for DeepInfra Inference API
      javascriptapiwrappertypescriptdeep-learningapi-clientllmllm-inference
      TypeScript
      •
      MIT License
      •1•10•4•2•Updated May 13, 2024May 13, 2024

    • lm-evaluation-harness

      Public
      A framework for few-shot evaluation of language models.
      Python
      •
      MIT License
      •2.1k•0•0•0•Updated Apr 29, 2024Apr 29, 2024

    • langchain

      Public
      ⚡ Building applications with LLMs through composability ⚡
      Python
      •
      MIT License
      •16k•1•0•0•Updated Jan 22, 2024Jan 22, 2024

    • litellm

      Public
      Call all LLM APIs using the OpenAI format. Use Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate (100+ LLMs)
      Python
      •
      MIT License
      •2.2k•0•0•0•Updated Jan 8, 2024Jan 8, 2024

    • text-generation-inference

      Public
      Large Language Model Text Generation Inference
      Python
      •
      Apache License 2.0
      •1.2k•9•0•6•Updated Dec 15, 2023Dec 15, 2023

    • fetch-stream-parser

      Public
      fetch-stream
      JavaScript
      •
      Apache License 2.0
      •1•0•0•0•Updated Nov 6, 2023Nov 6, 2023

    • fetch-event-source

      Public
      A better API for making Event Source requests, with all the features of fetch()
      TypeScript
      •
      MIT License
      •150•0•0•0•Updated Aug 18, 2023Aug 18, 2023

    • cog

      Public
      Containers for machine learning
      Go
      •
      Apache License 2.0
      •590•0•0•0•Updated Aug 1, 2023Aug 1, 2023

    • cog-llama-2

      Public
      A cog for running llama-2 using llama.cpp server
      Python
      •0•0•0•0•Updated Aug 1, 2023Aug 1, 2023

    • transformers

      Public
      🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
      Python
      •
      Apache License 2.0
      •28k•0•0•0•Updated Jul 24, 2023Jul 24, 2023

    • superfans-gpu-controller

      Public
      NVIDIA GPU-based FAN controller for SUPERMICRO server
      Python
      •
      MIT License
      •3•0•0•0•Updated Apr 25, 2023Apr 25, 2023

    • whisper-timestamped

      Public
      Multilingual Automatic Speech Recognition with word-level timestamps and confidence
      Python
      •
      GNU Affero General Public License v3.0
      •171•0•0•0•Updated Mar 7, 2023Mar 7, 2023

    • sentence-transformers

      Public
      Multilingual Sentence & Image Embeddings with BERT
      Python
      •
      Apache License 2.0
      •2.6k•0•0•0•Updated Feb 28, 2023Feb 28, 2023

    • full-stack-deep-learning-website

      Public
      Source for https://fullstackdeeplearning.com
      HTML
      •207•0•0•0•Updated Feb 14, 2023Feb 14, 2023