Skip to content

PyPI v0.9.0
Compare
Choose a tag to compare
@clearml-bot clearml-bot released this 21 Mar 15:38
· 99 commits to main since this release
0.9.0
c83403c

Redesign Release

Notice: This release is not backwards compatible

  • Easy to deploy & configure
    • Support Machine Learning Models (Scikit Learn, XGBoost, LightGBM)
    • Support Deep Learning Models (Tensorflow, PyTorch, ONNX)
    • Customizable RestAPI for serving (i.e. allow per model pre/post-processing for easy integration)
  • Flexible
    • On-line model deployment
    • On-line endpoint model/version deployment (i.e. no need to take the service down)
    • Per model standalone preprocessing and postprocessing python code
  • Scalable
    • Multi model per container
    • Multi models per serving service
    • Multi-service support (fully separated multiple serving service running independently)
    • Multi cluster support
    • Out-of-the-box node auto-scaling based on load/usage
  • Efficient
    • multi-container resource utilization
    • Support for CPU & GPU nodes
    • Auto-batching for DL models
  • Automatic deployment
    • Automatic model upgrades w/ canary support
    • Programmable API for model deployment
  • Canary A/B deployment
    • Online Canary updates
  • Model Monitoring
    • Usage Metric reporting
    • Metric Dashboard
    • Model performance metric
    • Model performance Dashboard

Features:

  • FastAPI integration for inference service
  • multi-process Gunicorn for inference service
  • Dynamic preprocess python code loading (no need for container/process restart)
  • Model files download/caching (http/s3/gs/azure)
  • Scikit-learn. XGBoost, LightGBM integration
  • Custom inference, including dynamic code loading
  • Manual model upload/registration to model repository (http/s3/gs/azure)
  • Canary load balancing
  • Auto model endpoint deployment based on model repository state
  • Machine/Node health metrics
  • Dynamic online configuration
  • CLI configuration tool
  • Nvidia Triton integration
  • GZip request compression
  • TorchServe engine integration
  • Prebuilt Docker containers (dockerhub)
  • Docker-compose deployment (CPU/GPU)
  • Scikit-Learn example
  • XGBoost example
  • LightGBM example
  • PyTorch example
  • TensorFlow/Keras example
  • Model ensemble example
  • Model pipeline example
  • Statistics Service
  • Kafka install instructions
  • Prometheus install instructions
  • Grafana install instructions
Assets 2
Loading