Migrate docs/ (#3243)

docs/source/conf.py

@@ -54,9 +54,6 @@
     "autosectionlabel.*",
 ]
 
-# Add any paths that contain templates here, relative to this directory.
-templates_path = ["_templates"]
-
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.

docs/source/guide/explanation/additional_features/auto_configuration.rst

@@ -25,7 +25,7 @@ Basically, to start the training and obtain a good baseline with the best trade-
             (otx) ...$ otx train ... --data_root <path_to_data_root>
 
 
-After dataset preparation, the training will be started with the middle-sized template to achieve competitive accuracy preserving fast inference.
+After dataset preparation, the training will be started with the middle-sized recipe to achieve competitive accuracy preserving fast inference.
 
 
 Supported dataset formats for each task:

docs/source/guide/explanation/additional_features/class_incremental_sampler.rst

@@ -0,0 +1,58 @@
+Class-Incremental Sampler
+===========================
+
+This sampler is a sampler that creates an effective batch.
+For default setting, the square root of (number of old data/number of new data) is used as the ratio of old data.
+
+.. tab-set::
+
+    .. tab-item:: API
+
+        .. code-block:: python
+
+            from otx.algo.samplers.class_incremental_sampler import ClassIncrementalSampler
+
+            dataset = OTXDataset(...)
+            class_incr_sampler = ClassIncrementalSampler(
+                dataset=dataset,
+                batch_size=32,
+                old_classes=["car", "truck"],
+                new_classes=["bus"],
+            )
+
+    .. tab-item:: CLI
+
+        .. code-block:: shell
+
+            (otx) ...$ otx train ... \
+                                 --data.config.train_subset.sampler.class_path otx.algo.samplers.class_incremental_sampler.ClassIncrementalSampler \
+                                 --data.config.train_subset.sampler.init_args.old_classes '[car,truck]' \
+                                 --data.config.train_subset.sampler.init_args.new_classes '[bus]'
+
+
+Balanced Sampler
+===========================
+
+This sampler is a sampler that creates an effective batch.
+It helps ensure balanced sampling by class based on the distribution of class labels during supervised learning.
+
+
+.. tab-set::
+
+    .. tab-item:: API
+
+        .. code-block:: python
+
+            from otx.algo.samplers.balanced_sampler import BalancedSampler
+
+            dataset = OTXDataset(...)
+            class_incr_sampler = BalancedSampler(
+                dataset=dataset,
+            )
+
+    .. tab-item:: CLI
+
+        .. code-block:: shell
+
+            (otx) ...$ otx train ... \
+                                 --data.config.train_subset.sampler.class_path otx.algo.samplers.balanced_sampler.BalancedSampler

docs/source/guide/explanation/additional_features/hpo.rst

@@ -55,61 +55,110 @@ ASHA also includes a technique called Hyperband, which is used to determine how
 How to configure hyper-parameter optimization
 *********************************************
 
-You can configure HPO by modifying the ``hpo_config.yaml`` file. This file contains everything related to HPO, including the hyperparameters to optimize, the HPO algorithm, and more. The ``hpo_config.yaml`` file already exists with default values in the same directory where ``template.yaml`` resides. Here is the default ``hpo_config.yaml`` file for classification:
+You can configure HPO using argument named *hpo_config*.
+It's *HpoConfig* dataclass which inlcudes hyperparameters to optimize, expected time ratio and more.
+If HPO is executed without confgiguration, default value of *HpoConfig* is used and learning rate and batch size are set as hyper parameterse to optimize.
+Here is how to configure using *hpo_config*.
+
+.. tab-set::
+
+    .. tab-item:: API
+
+        .. code-block:: python
+
+            from otx.engine import Engine
+            from otx.core.config.hpo import HpoConfig
+
+            hpo_config = HpoConfig(
+                search_space={
+                    "optimizer.lr" : {
+                        "type" : "uniform",
+                        "min" : 0.001,
+                        "max" : 0.1,
+                    }
+                },
+                expected_time_ratio=6,
+            )
+
+            engine = Engine(data_root="<path_to_data_root>")
+            engine.train(run_hpo=True, hpo_config=hpo_config)
+
+    .. tab-item:: CLI
+
+        .. code-block:: shell
+
+            (otx) ...$ otx train ... --run_hpo True  --hpo_config.expected_time_ratio 6
 
-.. code-block::
 
-    Need to Update!
+As above, you can set HPO configuration by both API and CLI. Including ones here, you can configure various parameters of HPO.
+You can configure almost parameters using CLI except search space of hyper parameters.
+But search space requires dictionray format, so it can be set only on API.
+Here is explanation of all HPO configuration.
 
-As you can see, there are a few attributes required to run HPO.
-Fortunately, there are not many attributes, so it's not difficult to write your own ``hpo_config.yaml`` file. The more detailed description is as follows:
 
-- **hp_space** (*List[Dict[str, Any]]*, `required`) - Hyper parameter search space to find. It should be list of dictionary. Each dictionary has a hyperparameter name as the key and param_type and range as the values. You can optimize any learning parameters of each task.
+- **search_space** (*list[dict[str, Any]]*, `required`) - Hyper parameter search space to find. It should be list of dictionary. Each dictionary has a hyperparameter name as the key and param_type and range as the values. You can optimize any learning parameters of each task.
 
   - **Keys of each hyper parameter**
 
-    - **param_type** (*str*, `required`) : Hyper parameter search space type. It must be one of the following:
+    - **type** (*str*, `required`) : Hyper parameter search space type. It must be one of the following:
 
       - uniform : Samples a float value uniformly between the lower and upper bounds.
       - quniform : Samples a quantized float value uniformly between the lower and upper bounds.
       - loguniform : Samples a float value after scaling search space by logarithm scale.
       - qloguniform : Samples a quantized float value after scaling the search space by logarithm scale.
       - choice : Samples a categorical value.
 
-    - **range** (*List[Any]*, `required`)
+    - **range** (*list*, `required`)
 
-      - uniform : List[Union[float, int]]
+      - uniform : list[float | int]
 
-        - min (*Union[float, int]*, `required`) : The lower bound of search space.
-        - max (*Union[float, int]*, `required`) : The upper bound of search space.
+        - min (*float | int*, `required`) : The lower bound of search space.
+        - max (*float | int*, `required`) : The upper bound of search space.
 
-      - quniform : List[Union[float, int]]
+      - quniform : list[float | int]
 
-        - min (*Union[float, int]*, `required`) : The lower bound of search space.
-        - max (*Union[float, int]*, `required`) : The upper bound of search space.
-        - step (*Union[float, int]*, `required`) : The unit value of search space.
+        - min (*float | int*, `required`) : The lower bound of search space.
+        - max (*float | int*, `required`) : The upper bound of search space.
+        - step (*float | int*, `required`) : The unit value of search space.
 
-      - loguniform : List[Union[float, int])
+      - loguniform : list[float | int]
 
-        - min (*Union[float, int]*, `required`) : The lower bound of search space.
-        - max (*Union[float, int]*, `required`) : The upper bound of search space.
-        - log_base (*Union[float, int]*, *default=10*) : The logarithm base.
+        - min (*float | int*, `required`) : The lower bound of search space.
+        - max (*float | int*, `required`) : The upper bound of search space.
+        - log_base (*float | int*, *default=10*) : The logarithm base.
 
       - qloguniform : List[Union[float, int]]
 
-        - min (*Union[float, int]*, `required`) : The lower bound of search space
-        - max (*Union[float, int]*, `required`) : The upper bound of search space
-        - step (*Union[float, int]*, `required`) : The unit value of search space
-        - log_base (*Union[float, int]*, *default=10*) : The logarithm base.
+        - min (*float | int*, `required`) : The lower bound of search space
+        - max (*float | int*, `required`) : The upper bound of search space
+        - step (*float | int*, `required`) : The unit value of search space
+        - log_base (*float | int*, *default=10*) : The logarithm base.
+
+      - choice : *list | tuple*
+
+        - vaule : values to choose as candidates.
+
+
+- **save_path** (*str*, *default='None'*) Path to save a HPO result.
+
+- **mode** (*str*, *default='max'*) - Optimization mode for the metric. It determines whether the metric should be maximized or minimized. The possible values are 'max' and 'min', respectively.
+
+- **num_workers** (*int*, *default=1*) How many trials will be executed in parallel.
+
+- **expected_time_ratio** (*int*, *default=4*) How many times to use for HPO compared to training time.
+
+- **maximum_resource** (*int*, *default=None*) - Maximum number of training epochs for each trial. When the training epochs reaches this value, the trial stop to train.
+
+- **minimum_resource** (*int*, *default=None*) - Minimum number of training epochs for each trial. Each trial will run at least this epochs, even if the performance of the model is not improving.
 
-      - choice : List[Any]
+- **prior_hyper_parameters** (*dict | list[dict]*, *default=None*) Hyper parameters to try first.
 
-        - vaule : values to be chosen from candidates.
+- **acceptable_additional_time_ratio** (*float | int*, *default=1.0*) How much ratio of additional time is acceptable.
 
-- **metric** (*str*, *default='mAP*') - Name of the metric that will be used to evaluate the performance of each trial. The hyperparameter optimization algorithm will aim to maximize or minimize this metric depending on the value of the mode hyperparameter. The default value is 'mAP'.
+- **reduction_factor** (*int*, *default=3*) How many trials to promote to next rung. Only top 1 / reduction_factor of rung trials can be promoted.
 
-- **mode** (*str*, *default='max*') - Optimization mode for the metric. It determines whether the metric should be maximized or minimized. The possible values are 'max' and 'min', respectively. The default value is 'max'.
+- **asynchronous_bracket** (*bool*, *default=True*) Whether to operate SHA asynchronously.
 
-- **maximum_resource** (*int*, *default=None*) - Maximum number of training epochs for each trial. When the number of training epochs reaches this value, the training of the trial will stop. The default value is None.
+- **asynchronous_sha** (*bool*, *default=True*) Whether SHAs(brackets) are running parallelly or not.
 
-- **minimum_resource** (*int*, *default=None*) - Minimum number of training epochs for each trial. Each trial will run for at least this many epochs, even if the performance of the model is not improving. The default value is None.
+**reduction_factor**, **asynchronous_bracket** and **asynchronous_sha** are HyperBand hyper parameters. If you want to know them more, please refer `ASHA <https://arxiv.org/pdf/1810.05934.pdf>`_.

docs/source/guide/explanation/additional_features/index.rst

@@ -13,3 +13,4 @@ Additional Features
    xai
    fast_data_loading
    tiling
+   class_incremental_sampler

docs/source/guide/explanation/additional_features/tiling.rst

@@ -67,12 +67,6 @@ To enable tiling in OTX training, set ``data.config.tile_config.enable_tiler`` p
 
             (otx) ...$ otx train ... --data.config.tile_config.enable_tiler True
 
-.. note::
-
-    To learn how to deploy the trained model and run the exported demo, refer to :doc:`../../tutorials/base/deploy`.
-
-    To learn how to run the demo in CLI and visualize results, refer to :doc:`../../tutorials/base/demo`.
-
 
 Tile Size and Tile Overlap Optimization
 -----------------------------------------