From 12042efe01d94d813c44bdbd2902966804b20c56 Mon Sep 17 00:00:00 2001 From: Andrey Velichkevich Date: Wed, 10 May 2023 12:41:20 +0100 Subject: [PATCH 1/4] Add Katib ROADMAP 2022/2023 --- ROADMAP.md | 42 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index f10a5c52873..754568655b7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,3 +1,43 @@ +# Katib 2022/2023 Roadmap + +## AutoML Features + +- Support advance HyperParameter tuning algorithms: + + - Population Based Training - [#1382](https://github.com/kubeflow/katib/issues/1382) + - Tree of Parzen Estimators (TPE) + - Multivariate TPE + - Sobol’s Quasirandom Sequence + - Asynchronous Successive Halving - [ASHA](https://arxiv.org/pdf/1810.05934.pdf) + +- Support Auto Model Compression - [#460](https://github.com/kubeflow/katib/issues/460) +- Support Auto Feature Engineering - [#475](https://github.com/kubeflow/katib/issues/475) +- Improve Neural Architecture Search design. + +## Backend and API Enhancements + +- Conformance tests for Katib - [#2044](https://github.com/kubeflow/katib/issues/2044) +- Support various HP distributions (log-uniform, uniform, normal) - [#1207](https://github.com/kubeflow/katib/issues/1207) +- Support push-based metrics collection in Katib - [#577](https://github.com/kubeflow/katib/issues/577) +- Support PostgreSQL as a Katib DB - [#915](https://github.com/kubeflow/katib/issues/915) +- Promote Katib CRDs (`Experiment`, `Suggestion`, and `Trial`) to the `v1` version +- Support multiple CRD versions (`v1beta1`, `v1`) with conversion webhook + +## Improve Katib User Experience + +- Simplify Katib Experiment creation with Katib SDK - [#1951](https://github.com/kubeflow/katib/pull/1951) +- Fully migrate to a new Katib UI - [Project 1](https://github.com/kubeflow/katib/projects/1) +- Expose Trial logs in Katib UI - [#971](https://github.com/kubeflow/katib/issues/971) +- Enhance Katib UI visualization metrics for AutoML Experiments +- Improve Katib Config UX - [#2150](https://github.com/kubeflow/katib/issues/2150) + +## Integration with Kubeflow Components + +- Kubeflow Pipeline as a Katib Trial target - [#1914](https://github.com/kubeflow/katib/issues/1914) +- Improve data passing when Katib Experiment is part of Kubeflow Pipeline - [#1846](https://github.com/kubeflow/katib/issues/1846) + +# History + # Katib 2021 Roadmap ## New Features @@ -24,8 +64,6 @@ - Support multiple CRD version with conversion webhook - MLMD integration with Katib Experiments -# History - # Katib 2020 Roadmap ## New Features From a969bf4ef9d971af02244ead7de792e9506b7a99 Mon Sep 17 00:00:00 2001 From: Andrey Velichkevich Date: Wed, 10 May 2023 12:52:22 +0100 Subject: [PATCH 2/4] Add multi-objective optimization --- ROADMAP.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 754568655b7..cbae77c97f1 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -4,20 +4,21 @@ - Support advance HyperParameter tuning algorithms: - - Population Based Training - [#1382](https://github.com/kubeflow/katib/issues/1382) + - Population Based Training (PBT) - [#1382](https://github.com/kubeflow/katib/issues/1382) - Tree of Parzen Estimators (TPE) - Multivariate TPE - Sobol’s Quasirandom Sequence - Asynchronous Successive Halving - [ASHA](https://arxiv.org/pdf/1810.05934.pdf) +- Support multi-objective optimization - [#1549](https://github.com/kubeflow/katib/issues/1549) +- Support various HP distributions (log-uniform, uniform, normal) - [#1207](https://github.com/kubeflow/katib/issues/1207) - Support Auto Model Compression - [#460](https://github.com/kubeflow/katib/issues/460) - Support Auto Feature Engineering - [#475](https://github.com/kubeflow/katib/issues/475) -- Improve Neural Architecture Search design. +- Improve Neural Architecture Search design ## Backend and API Enhancements - Conformance tests for Katib - [#2044](https://github.com/kubeflow/katib/issues/2044) -- Support various HP distributions (log-uniform, uniform, normal) - [#1207](https://github.com/kubeflow/katib/issues/1207) - Support push-based metrics collection in Katib - [#577](https://github.com/kubeflow/katib/issues/577) - Support PostgreSQL as a Katib DB - [#915](https://github.com/kubeflow/katib/issues/915) - Promote Katib CRDs (`Experiment`, `Suggestion`, and `Trial`) to the `v1` version From 5e49dc33f6c8c9fc68482022a7e356e11882fb1c Mon Sep 17 00:00:00 2001 From: Andrey Velichkevich Date: Wed, 24 May 2023 17:07:57 +0100 Subject: [PATCH 3/4] Add Scalability Improvements --- ROADMAP.md | 1 + 1 file changed, 1 insertion(+) diff --git a/ROADMAP.md b/ROADMAP.md index cbae77c97f1..4f29f55e1d3 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -21,6 +21,7 @@ - Conformance tests for Katib - [#2044](https://github.com/kubeflow/katib/issues/2044) - Support push-based metrics collection in Katib - [#577](https://github.com/kubeflow/katib/issues/577) - Support PostgreSQL as a Katib DB - [#915](https://github.com/kubeflow/katib/issues/915) +- Improve Katib scalability - [#1847](https://github.com/kubeflow/katib/issues/1847) - Promote Katib CRDs (`Experiment`, `Suggestion`, and `Trial`) to the `v1` version - Support multiple CRD versions (`v1beta1`, `v1`) with conversion webhook From 6357e3779d4a6329866228c91338676f7b0e1472 Mon Sep 17 00:00:00 2001 From: Andrey Velichkevich Date: Thu, 24 Aug 2023 14:02:52 +0100 Subject: [PATCH 4/4] Remove Katib CRD naming --- ROADMAP.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index 4f29f55e1d3..565875b1913 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -22,7 +22,7 @@ - Support push-based metrics collection in Katib - [#577](https://github.com/kubeflow/katib/issues/577) - Support PostgreSQL as a Katib DB - [#915](https://github.com/kubeflow/katib/issues/915) - Improve Katib scalability - [#1847](https://github.com/kubeflow/katib/issues/1847) -- Promote Katib CRDs (`Experiment`, `Suggestion`, and `Trial`) to the `v1` version +- Promote Katib APIs to the `v1` version - Support multiple CRD versions (`v1beta1`, `v1`) with conversion webhook ## Improve Katib User Experience