Motivation

Adan optimizer is a big hit recently, good feature to support for meta learning research

Resource

• paper: https://arxiv.org/abs/2208.06677
• source code: https://github.com/sail-sg/Adan

Roadmap

• low-level alias
• high-level Optimizer
• high-level MetaOptimizer
• low-level test
• high-level Optimizer test
• api docs

Checklist

• I have checked that there is no similar issue in the repo (required)