Code

ZeRO stage 1 refresh#1042

Code

Merged

jeffra merged 12 commits intomasterdeepspeedai/DeepSpeed:masterfrom jeffra/z1-refresh-4deepspeedai/DeepSpeed:jeffra/z1-refresh-4Copy head branch name to clipboardMay 19, 2021

+495-397

Commits

Commits on May 4, 2021

• add zero-1 mode to zero-2 codebase

  jeffra

  committed
  e344463

  Copy full SHA for e344463

• fixes to support PP

  jeffra

  committed
  a2fb18a

  Copy full SHA for a2fb18a

Commits on May 5, 2021

• update json docs to add grad_hooks param

  jeffra

  committed
  10eda14

  Copy full SHA for 10eda14

• Merge branch 'master' into jeffra/z1-refresh-4

  jeffra

  authored
  7ba9c96

  Copy full SHA for 7ba9c96

Commits on May 10, 2021

• Merge branch 'master' into jeffra/z1-refresh-4

  jeffra

  authored
  d994f88

  Copy full SHA for d994f88

Commits on May 13, 2021

• Merge branch 'master' into jeffra/z1-refresh-4

  jeffra

  authored
  faf3b3a

  Copy full SHA for faf3b3a

• add missing comma in merge confict fix

  jeffra

  committed
  df795ca

  Copy full SHA for df795ca

Commits on May 14, 2021

• remove grad hooks, unify grad reduce from engine

  jeffra

  committed
  a9ef599

  Copy full SHA for a9ef599

• Merge branch 'master' into jeffra/z1-refresh-4

  jeffra

  authored
  bcd3647

  Copy full SHA for bcd3647

Commits on May 17, 2021

• fixes to support PP

  jeffra

  committed
  567339e

  Copy full SHA for 567339e

• Merge branch 'master' into jeffra/z1-refresh-4

  jeffra

  authored
  4e6f92b

  Copy full SHA for 4e6f92b

Commits on May 19, 2021

• Merge branch 'master' into jeffra/z1-refresh-4

  jeffra

  authored
  d11285f

  Copy full SHA for d11285f