DVCLive API code snippet in notebooks

[tibor-mach]

In the extension -> setup -> experiments we have the following example under Python API:

import random
import sys
from dvclive import Live

with Live(save_dvc_exp=True) as live:
    epochs = int(sys.argv[1])
    live.log_param("epochs", epochs)
    for epoch in range(epochs):
        live.log_metric("train/accuracy", epoch + random.random())
        live.log_metric("train/loss", epochs - epoch - random.random())
        live.log_metric("val/accuracy",epoch + random.random() )
        live.log_metric("val/loss", epochs - epoch - random.random())
        live.next_step()

This works nicely if you copy it to a script and run it from the command line. But if people expect to use this in a jupyter notebook (which they might with DVCLive) and then copy it to a cell and run it, this will fail since no arguments are received.

Maybe it would make sense to create an example which can be run in notebooks also? Just replacing sys.argv[1] with a positive integer constant should do it.

[mattseddon]

See also https://dvc.org/doc/start/experiments/experiment-tracking?tab=General-Python-API#running

[mattseddon]

The same snippet is also in Studio: https://github.com/iterative/studio/blob/492ba930d81cf8460c368286b26df4cb67957c07/frontend/src/composite/ProjectExperimentingGuide/GetStartedTabs.tsx#L30

[mattseddon]

@tibor-mach there is more context/discussion in https://github.com/iterative/studio/issues/6422 / iterative/dvclive#635 PTAL. If you have strong opinions contributions are welcome. Code for the extension is here:

vscode-dvc/webview/src/setup/snippets/pythonApi.py

Line 1 in c0cc75c

import random

and dvc.org is here:

https://github.com/iterative/dvc.org/blob/main/content/docs/start/experiments/experiment-tracking.md#L92

Studio link is shown above.

[tibor-mach]

Thanks @mattseddon !

I agree with @omesser (this comment) especially on the sys.argv.

The first thing I tried without thinking about the snippet (or even really reading it, frankly) was to copy-paste it into a notebook and run it.

Obviously, it fails and with an error which is confusing at first (only at this point did I start actually reading the code of the snippet).

Personally I would just go with Oded's example and perhaps just slightly simplify it for compactness in VS code.

WDYT?

If you agree I can make a PR (probably sometimes next week though)

[dberenbaum]

@tibor-mach I haven't seen that anyone else has strong opinions on the snippets, so I think you could make those changes and they will likely be approved without too much discussion. I think the main reason the previous discussion died was that it may not be that high impact to be worth the effort. If you have the time and there's nobody who is opposed to the changes, feel free to go for it!

[mattseddon]

@tibor-mach it doesn't seem like this is a priority. Closing for now.