[Refactor] Refactor projects folder

projects/example_project/README.md

@@ -5,8 +5,13 @@ This is an example README for community `projects/`. We have provided detailed e
 - [Dummy MAE Wrapper](#dummy-mae-wrapper)
   - [Description](#description)
   - [Usage](#usage)
-    - [Pre-training commands](#pre-training-commands)
-    - [Downstream tasks commands](#downstream-tasks-commands)
+    - [Setup Environment](#setup-environment)
+    - [Data Preparation](#data-preparation)
+    - [Pre-training Commands](#pre-training-commands)
+      - [On Local Single GPU](#on-local-single-gpu)
+      - [On Multiple GPUs](#on-multiple-gpus)
+      - [On Multiple GPUs with Slurm](#on-multiple-gpus-with-slurm)
+    - [Downstream Tasks Commands](#downstream-tasks-commands)
   - [Results](#results)
   - [Citation](#citation)
   - [Checklist](#checklist)
@@ -21,27 +26,105 @@ This project implements a dummy MAE wrapper, which prints "Welcome to MMSelfSup"
 
 ## Usage
 
-<!-- For a typical model, this section should contain the commands for training and testing. You are also suggested to dump your environment specification to env.yml by `conda env export > env.yml`. -->
+<!-- For a typical model, this section should contain the commands for dataset prepareation, pre-training, downstream tasks. You are also suggested to dump your environment specification to env.yml by `conda env export > env.yml`. -->
 
-### Pre-training commands
+### Setup Environment
 
-In MMSelfSup's root directory, run the following command to train the model:
+Please refer to [Get Started](https://mmselfsup.readthedocs.io/en/1.x/get_started.html) documentation of MMSelfSup.
+
+### Data Preparation
+
+To show the dataset directory or provide the commands for dataset preparation if needed.
+
+For example:
+
+```text
+data/
+└── imagenet
+    ├── train
+    ├── val
+    └── meta
+        ├── train.txt
+        └── val.txt
+```
+
+### Pre-training Commands
+
+At first, you need to add the current folder to `PYTHONPATH`, so that Python can find your model files. In `example_project/` root directory, please run command below to add it.
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+Then run the following commands to train the model:
+
+#### On Local Single GPU
 
 ```bash
-python tools/train.py projects/example_project/configs/dummy-mae_vit-base-p16_8xb512-amp-coslr-300e_in1k.py
+mim train mmselfsup $CONFIG --work-dir $WORK_DIR
+
+# a specific command example
+mim train mmselfsup configs/dummy-mae_vit-base-p16_8xb512-amp-coslr-300e_in1k.py \
+    --work-dir work_dirs/dummy_mae/
 ```
 
-### Downstream tasks commands
+#### On Multiple GPUs
 
-In MMSelfSup's root directory, run the following command to train the downstream model:
+```bash
+# a specific command examples, 8 GPUs here
+mim train mmselfsup configs/dummy-mae_vit-base-p16_8xb512-amp-coslr-300e_in1k.py \
+    --work-dir work_dirs/dummy_mae/ \
+    --launcher pytorch --gpus 8
+```
+
+Note:
+
+- CONFIG: the config files under the directory `configs/`
+- WORK_DIR: the working directory to save configs, logs, and checkpoints
+
+#### On Multiple GPUs with Slurm
 
 ```bash
-sh tools/benchmarks/classification/mim_dist_train.sh ${CONFIGS} ${CHECKPOINT} [optional args]
+# a specific command example: 16 GPUs in 2 nodes
+mim train mmselfsup configs/dummy-mae_vit-base-p16_8xb512-amp-coslr-300e_in1k.py \
+    --work-dir work_dirs/dummy_mae/ \
+    --launcher slurm --gpus 16 --gpus-per-node 8 \
+    --partition $PARTITION
+```
 
-# the example of custom command
-GPUS=1 sh tools/benchmarks/classification/mim_dist_train.sh projects/example_projects/configs/xxx.py ${CHECKPOINT} --work-dir work_dirs/example_projects/classification/
+Note:
+
+- CONFIG: the config files under the directory `configs/`
+- WORK_DIR: the working directory to save configs, logs, and checkpoints
+- PARTITION: the slurm partition you are using
+
+### Downstream Tasks Commands
+
+In MMSelfSup's root directory, run the following command to train the downstream model:
+
+```bash
+mim train mmcls $CONFIG \
+    --work-dir $WORK_DIR \
+    --launcher pytorch -gpus 8 \
+    [optional args]
+
+# a specific command example
+mim train mmcls configs/vit-base-p16_ft-8xb128-coslr-100e_in1k.py \
+    --work-dir work_dirs/dummy_mae/classification/
+    --launcher pytorch -gpus 8 \
+    --cfg-options model.backbone.init_cfg.type=Pretrained \
+    model.backbone.init_cfg.checkpoint=https://download.openmmlab.com/mmselfsup/1.x/mae/mae_vit-base-p16_8xb512-fp16-coslr-300e_in1k/mae_vit-base-p16_8xb512-coslr-300e-fp16_in1k_20220829-c2cf66ba.pth \
+    model.backbone.init_cfg.prefix="backbone." \
+    $PY_ARGS
 ```
 
+Note:
+
+- CONFIG: the config files under the directory `configs/`
+- WORK_DIR: the working directory to save configs, logs, and checkpoints
+- CHECKPOINT: the pretrained checkpoint of MMSelfSup saved in working directory, like `$WORK_DIR/epoch_300.pth`
+- PY_ARGS: other optional args
+
 ## Results
 
 <!-- List the results as usually done in other model's README. [Example](https://github.com/open-mmlab/mmselfsup/blob/1.x/configs/selfsup/mae/README.md#models-and-benchmarks)
@@ -91,8 +174,12 @@ The Linear Eval and Fine-tuning results are based on ImageNet dataset.
 
 ## Checklist
 
-<!-- Here is a checklist illustrating a usual development workflow of a successful project, and also serves as an overview of this project's progress. The PIC (person in charge) or contributors of this project should check all the items that they believe have been finished, which will further be verified by codebase maintainers via a PR.
+Here is a checklist illustrating a usual development workflow of a successful project, and also serves as an overview of this project's progress.
+
+<!--The PIC (person in charge) or contributors of this project should check all the items that they believe have been finished, which will further be verified by codebase maintainers via a PR.
+
 OpenMMLab's maintainer will review the code to ensure the project's quality. Reaching the first milestone means that this project suffices the minimum requirement of being merged into 'projects/'. But this project is only eligible to become a part of the core package upon attaining the last milestone.
+
 Note that keeping this section up-to-date is crucial not only for this project's developers but the entire community, since there might be some other contributors joining this project and deciding their starting point from this list. It also helps maintainers accurately estimate time and effort on further code polishing, if needed.
 A project does not necessarily have to be finished in a single PR, but it's essential for the project to at least reach the first milestone in its very first PR. -->
 
@@ -138,4 +225,4 @@ A project does not necessarily have to be finished in a single PR, but it's esse
 
     <!-- It will be parsed by MIM and Inferencer. [Example](https://github.com/open-mmlab/mmselfsup/blob/1.x/configs/selfsup/mae/metafile.yml). In particular, you may have to refactor this README into a standard one. [Example](https://github.com/open-mmlab/mmselfsup/blob/1.x/configs/selfsup/mae/README.md) -->
 
-  - [ ] Refactor and Move your modules into the core package following the codebase's file hierarchy structure.
+- [ ] Refactor and Move your modules into the core package following the codebase's file hierarchy structure.

projects/example_project/configs/dummy-mae_vit-base-p16_8xb512-amp-coslr-300e_in1k.py

@@ -1,7 +1,7 @@
 _base_ = [
-    '../../../configs/selfsup/mae/mae_vit-base-p16_8xb512-amp-coslr-300e_in1k.py'  # noqa:E501
+    'mmselfsup::selfsup/mae/mae_vit-base-p16_8xb512-amp-coslr-300e_in1k.py'
 ]
 
-custom_imports = dict(imports=['projects.example_project'])
+custom_imports = dict(imports=['models'])
 
 _base_.model.type = 'DummyMAE'

projects/example_project/configs/vit-base-p16_ft-8xb128-coslr-100e_in1k.py

@@ -0,0 +1,124 @@
+# mmcls:: means we use the default settings from MMClassification
+_base_ = [
+    'mmcls::_base_/datasets/imagenet_bs64_swin_224.py',
+    'mmcls::_base_/schedules/imagenet_bs1024_adamw_swin.py',
+    'mmcls::_base_/default_runtime.py'
+]
+# MAE fine-tuning setting
+
+# model settings
+model = dict(
+    type='ImageClassifier',
+    backbone=dict(
+        type='VisionTransformer',
+        arch='base',
+        img_size=224,
+        patch_size=16,
+        drop_path_rate=0.1,
+        avg_token=True,
+        output_cls_token=False,
+        final_norm=False,
+        init_cfg=dict(type='Pretrained', checkpoint='')),
+    neck=None,
+    head=dict(
+        type='LinearClsHead',
+        num_classes=1000,
+        in_channels=768,
+        loss=dict(
+            type='LabelSmoothLoss', label_smooth_val=0.1, mode='original'),
+        init_cfg=[dict(type='TruncNormal', layer='Linear', std=2e-5)]),
+    train_cfg=dict(augments=[
+        dict(type='Mixup', alpha=0.8),
+        dict(type='CutMix', alpha=1.0)
+    ]))
+
+file_client_args = dict(backend='disk')
+
+train_pipeline = [
+    dict(type='LoadImageFromFile', file_client_args=file_client_args),
+    dict(
+        type='RandomResizedCrop',
+        scale=224,
+        backend='pillow',
+        interpolation='bicubic'),
+    dict(type='RandomFlip', prob=0.5, direction='horizontal'),
+    dict(
+        type='RandAugment',
+        policies='timm_increasing',
+        num_policies=2,
+        total_level=10,
+        magnitude_level=9,
+        magnitude_std=0.5,
+        hparams=dict(pad_val=[104, 116, 124], interpolation='bicubic')),
+    dict(
+        type='RandomErasing',
+        erase_prob=0.25,
+        mode='rand',
+        min_area_ratio=0.02,
+        max_area_ratio=0.3333333333333333,
+        fill_color=[103.53, 116.28, 123.675],
+        fill_std=[57.375, 57.12, 58.395]),
+    dict(type='PackClsInputs')
+]
+test_pipeline = [
+    dict(type='LoadImageFromFile', file_client_args=file_client_args),
+    dict(
+        type='ResizeEdge',
+        scale=256,
+        edge='short',
+        backend='pillow',
+        interpolation='bicubic'),
+    dict(type='CenterCrop', crop_size=224),
+    dict(type='PackClsInputs')
+]
+
+train_dataloader = dict(batch_size=128, dataset=dict(pipeline=train_pipeline))
+val_dataloader = dict(batch_size=128, dataset=dict(pipeline=test_pipeline))
+test_dataloader = val_dataloader
+
+# optimizer wrapper
+optim_wrapper = dict(
+    optimizer=dict(
+        type='AdamW',
+        lr=2e-3,
+        weight_decay=0.05,
+        eps=1e-8,
+        betas=(0.9, 0.999),
+        model_type='vit',  # layer-wise lr decay type
+        layer_decay_rate=0.65),  # layer-wise lr decay factor
+    constructor='mmselfsup.LearningRateDecayOptimWrapperConstructor',
+    paramwise_cfg=dict(
+        custom_keys={
+            '.ln': dict(decay_mult=0.0),
+            '.bias': dict(decay_mult=0.0),
+            '.cls_token': dict(decay_mult=0.0),
+            '.pos_embed': dict(decay_mult=0.0)
+        }))
+
+# learning rate scheduler
+param_scheduler = [
+    dict(
+        type='LinearLR',
+        start_factor=1e-4,
+        by_epoch=True,
+        begin=0,
+        end=5,
+        convert_to_iter_based=True),
+    dict(
+        type='CosineAnnealingLR',
+        T_max=95,
+        by_epoch=True,
+        begin=5,
+        end=100,
+        eta_min=1e-6,
+        convert_to_iter_based=True)
+]
+
+# runtime settings
+default_hooks = dict(
+    # save checkpoint per epoch.
+    checkpoint=dict(type='CheckpointHook', interval=1, max_keep_ckpts=3))
+
+train_cfg = dict(by_epoch=True, max_epochs=100)
+
+randomness = dict(seed=0, diff_rank_seed=True)