Bug: baby-llama fails

[sfadaei]

What happened?

Running baby-llama I get ggml.c:6845: GGML_ASSERT(false && "backwards pass not implemented") failed
I came across the following code in ggml_rope_back:

    if (a->grad) {
        GGML_ASSERT(false && "backwards pass not implemented");
        is_node = false;
    }

Since IMO the backward pass is already implemented, we need just to change this code to

    if (a->grad) {
        is_node = true;
    }

That should fix the baby-llama.
The backward pass of RoPE is present in ggml_compute_backward(..).
I can create a PR for this, or maybe I'm missing something?

Name and Version

version: 3827 (7691654)
built with Apple clang version 15.0.0 (clang-1500.3.9.4) for arm64-apple-darwin24.0.0

What operating system are you seeing the problem on?

Mac

Relevant log output

ggml.c:6845: GGML_ASSERT(false && "backwards pass not implemented") failed

[JohannesGaessler]

The error message means that there is no backward pass for ggml_rope_back. It's not actually needed but currently the backwards graph construction incorrectly allocates gradient tensors for the gradients (which are not needed, only the forward pass tensors need gradients). Should be fixed by ggml-org/ggml#966 but I'm currently heavily overhauling the training code in general.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.