Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

ROCK will need to have configurable notebook-dir to avoid data loss in Kubeflow #67

Open
kimwnasptd opened this issue Jan 25, 2024 · 0 comments

Comments

@kimwnasptd
Copy link

Creating an issue after our findings with @deusebio, for integrating the PySpark ROCK with Kubeflow.

In Kubeflow when a new Notebook Pod gets created the default behaviour is to mount a PVC under the, hardcoded, /home/jovyan. We'll need to have a mechanism for the ROCK to be able to configure the --notebook-dir arg of jupyterlab, so that we can set it accordingly for Kubeflow.

Example of how this is set in upstream
https://github.com/kubeflow/kubeflow/blob/master/components/example-notebook-servers/jupyter/s6/services.d/jupyterlab/run#L9

Making the code read an ENV variable should be good enough, and then from Kubeflow side we can create a PodDefault that would handle updating that variable

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant