v1.1rc1 testing (Chrome/MacOS)

[craig-willis]

Test Plan

• = tested
  Issue Issue identified
  Note Question/comment for future consideration

End-to-End Test

• Smoke test
• Open private session as user1
• https://girder.test.wholetale.org/api/v1/integration/zenodo?doi=10.5072%2Fzenodo.1021503&resource_server=sandbox.zenodo.org
• Start the interactive environment
• sh run.sh, confirm no errors
• Start a recorded run using run.sh, confirm SUCCESS
• Export tale
• Run locally
• Execute run.sh, confirm success
• Share the tale view-only with user2
• As user2, copy-on-launch. Confirm versions and runs were copied
• Start a recorded run using run.sh, confirm SUCCESS
• Export the tale
• sh run-local.sh and access the interactive environment
• In the interactive environment sh run.sh, confirm no errors

Issue: Globus transfer fails because globus-sdk is missing and keychain.json isn't configured -- also no indication that this needs to happen (whole-tale/whole-tale#115)
Issue: Error on DERIVA zip (whole-tale/whole-tale#114)
Note: Tale export/import doesn't retain permissions (e.g., execute bit on run.sh)

Splash page

• Splash page

1. Goto https://dashboard.test.wholetale.org
2. Confirm Terms of Service link
3. Confirm acknowledgements (NSF, XSEDE/Jetstream, Stata)
4. Confirm partner logos

Issue: TOS points to https://dashboard.test.wholetale.org/undefined/tos now noted in whole-tale/ngx-dashboard#255.
Note: Should we acknowledge Cornell, UNC, UMich?

Authentication

• Basic login flow

1. Goto https://dashboard.test.wholetale.org
2. Select "Access Whole Tale" button. You should be prompted to login via Globus Auth.
3. Confirm institutional login (OK)
4. Confirm Google login (OK)
5. Confirm ORCID login (NG)
6. In each case you should be taken to the "My Tales" page with your Gravatar displayed in the upper right

Issue: ORCID login fails with 500 error. Issue with Client ID not respecting requests from test.wholetale.org? Login is working on stage. (whole-tale/girder_wholetale#459)

• Basic logout flow

1. Select the "User" menu
2. Select the "Log out" menu item. You should be taken to the login page.
3. Select "Access Whole Tale" button.
4. Note that you are not prompted to login.
5. Select User menu > Logout.
6. Go to https://app.globus.org/logout to logout from Globus
7. Go to https://dashboard.test.wholetale.org
8. Select "Access Whole Tale". You should be prompted to login.

• Return-route for non-logged in users

1. Logout from Whole Tale
2. Attempt to access https://dashboard.test.wholetale.org/public
3. You should be redirected to https://dashboard.test.wholetale.org/#?rd=%252Fpublic
4. Select "Access Whole Tale".
5. You should be redirected to the "Public Tales" page.

• Forward auth

1. Create and start an empty tale instance
2. Copy the link
3. Start a new private session
4. Attempt to access the link
5. Login as user with permission on the tale. Confirm that the IDE displays.
6. Login as user without permission. Confirm instance access is denied.

• Analyze in WT (unauthenticated user)

1. Open new private session
2. Go to https://dashboard.test.wholetale.org/browse?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.820575&name=Automotive%20Sensor%20Data
3. Select "Access Whole Tale", login
4. Confirm "Create new Tale" dialog opens with title "Automotive Sensor Data" and datasource populated

Issue: whole-tale/ngx-dashboard#258. I must not have tested the fix thoroughly, because the create fails on the backend with RuntimeError: Lookup for "https:%2F%2Fdoi.org%2F10.5281%2Fzenodo.820575" failed with: Failed to interpret "https:%2F%2Fdoi.org%2F10.5281%2Fzenodo.820575" in any meaningful way. This suggests the URL isn't being correctly decoded. And there is no error except in the console. (whole-tale/ngx-dashboard#258)

Navigation

• General navigation

1. Select "Tale Dashboard" link. "My Tales" page should display
2. Click "Create New Tale" > "Create New Tale". The Create Tale modal should display. Click "Cancel" to close the modal
3. Select the "i" icon at the upper right corner; it should open the User Guide in new tab. Hovering should display tooltip.
4. Select the "Notification" button next to the "i" icon; it should display the notification panel. Hovering should display tooltip.
5. Select the "bug" icon; it should open https://github.com/whole-tale/whole-tale/issues in new tab (after authentication). Hovering should display tooltip.
6. Mouse over a Tale and select "View". The Run page should display
7. Click the "user" icon at the top right

• You should see a dropdown expand
• The dropdown should contain the user's first/last name and gravatar image
• The dropdown should provide a link to the Settings page
• The dropdown should provide an option to Logout
• Select the "Settings" option; it should bring you to the Settings page

Tale Dashboard

Preconditions:

• Assumes production Tales present (e.g., LIGO, Materials, etc).

• No running instances

• General

1. Select "My Tales" link. Confirm "My Tales" page displays
2. Confirm that no Tales are running
3. Select "Shared with Me" link. Confirm "Shared with Me" page displays,

• Search

1. Select "Public Tales" link. Confirm "Public Tales" page displays
2. Enter "LIGO" as search term. Confirm only LIGO tale displays (by K. Kowalik)
3. Clear search term. Confirm all tales display

• View tale

1. Select "View" button on LIGO Tale
2. Confirm that Metadata page displays by default
3. Confirm that you cannot delete or edit Tale properties
4. Select Back button. Confirm that you are taken back to Public Tales page
5. Select "View" button on Tale you own
   1. Confirm that the Run and Close buttons are present
   2. Confirm that you can edit Tale properties
   3. Select "Close"
   4. Confirm that you are taken back to "My Tales"

• Launch instance

1. Select "Run Tale" button on "Ligo" tale
   1. Confirm you are prompted to Copy and Run
   2. Select "Confirm"
   3. Confirm that you are redirected to the Tale's "Metadata" page
   4. Confirm that the "Edit" button is present
   5. Select "Run Tale"
   6. Confirm that the "Interact" page is displayed with spinner
   7. Confirm that the notification dialog appears
   8. Confirm that the step progression is correct
   9. Confirm that the "View Logs" button shows the build logs in a modal
   10. Navigate to the "My Tales" page
   11. Confirm that the Tale is shown under "Currently Running"
   12. Select "View" on the Tale card
   13. Confirm that "Interact" page is displayed
2. Select "Stop" to end the Tale instance
   1. Confirm that the Tale has stopped
   2. Confirm that the button has changed to Run Tale
3. Navigate to "My Tales"
   1. Confirm that there are no Tales under "Currently Running"
   2. Select the "Run Tale" button on a Tale you own
   3. After the instance is launched, navigate back to the My Tales page
   4. Confirm that you cannot select the delete button on the My Tales page
   5. Confirm that the "Run Tale" button has been replaced with "Stop Tale"
   6. Confirm that the Tale is shown at the top of the page under "Currently running"

Notes:

• View logs appeared to be hung while container logs said "exporting layers". Long build and push.

Managing Data

Preconditions

• Empty home directory

• No registered data

• Register General

1. Open an owned Tale
2. Select 'External Data' > "+" > Web (DOI or URL)
3. Confirm list of providers matches list of supported providers (Globus, DataONE, Dataverse, Zenodo)

Issue: Missing DERIVA

• Register DataONE data

1. Open an owned Tale
2. Select 'External Data' > "+" > Web (DOI or URL)
3. Confirm modal displays
4. Search for DOI 10.5065/D6862DM8
5. Confirm "Humans and Hydrology" dataset is found on DataONE.
6. Select Register
7. Confirm dataset appears in Manage Data list
8. Confirm additional URL formats:
   1. https://search.dataone.org/view/doi:10.5063/F1Z60M87
   2. https://search.dataone.org/view/https://pasta.lternet.edu/package/metadata/eml/knb-lter-nwt/115/1
   3. https://search.dataone.org/view/doi:10.18739/A2CN6XZ7H
   4. https://raw.githubusercontent.com/whole-tale/dashboard/master/.travis.yml

• Dataverse

1. Register dataset doi:10.7910/DVN/SZ9YXZ
2. Confirm "Replication Data for: Direct Democracy and Women's Political Engagement" from Dataverse

• Globus/MDF

1. Confirm DOI resolution for http://dx.doi.org/doi:10.18126/M2301J results in https://petreldata.net/mdf/detail/pub_30_shahani_twinmediated_v1.2
2. Register dataset https://petreldata.net/mdf/detail/pub_30_shahani_twinmediated_v1.2

• DERIVA

1. Register dataset https://pbcconsortium.s3.amazonaws.com/wholetale/5ad7cdf55b0d5007601015b7ff1ea8d6/2021-11-09_21.47.58/Dataset_1-882P.zip

Run Tale

Preconditions:

• No running Tale instances

• General

1. On "Public Tales", click "Run" on LIGO tale (Copy on Launch)
2. Select "Run Tale" to start the instance
3. Confirm you are redirected to the Run page
   1. Confirm that the Run panel displays the Tale icon, title, and author
   2. Confirm that the Run page displays the Interact, Files, Metadata and Share tabs
   3. Confirm that Interact tab is displayed by default
   4. Confirm notification panel displays and progression is correct
4. Select the "..." menu.
   1. Confirm that menu options exist for "Rebuild Tale", "Restart Tale", "Save Tale Version", "Recorded Run", "Duplicate Tale", "Publish Tale", "Export Tale", "Connect to Git Repository..."
   2. Confirm that the "Read the docs" link displays https://wholetale.readthedocs.io/en/latest/users_guide/run.html in a new tab or window (stable is for production)
   3. Confirm that the "View Fullscreen" link displays the Run panel in full screen

Test case: Missing recorded run from menu list

• Interact tab

1. Select the "Interact" tab
2. Confirm Jupyter environment is displayed in iframe
3. Hover over popout icon, confirm tooltip displays. (not yet!)
4. Select "popout" icon. Confirm Jupyter is opened in new tab,

• Files tab

1. Select "Files" tab
2. You should see a vertical navigation menu with two sections CURRENT and ARCHIVED.
3. Under CURRENT you should see three tabs: "Home", "External Data", and "Tale Workspace".
4. Under ARCHIVED you should see Saved Versions and Recorded Runs under ARCHIVED
5. Confirm Tale Workspace is selected by default

• Metadata tab - Owned Tale

1. Open a Tale that you own (e.g., LIGO Tale copy)
2. Navigate to the Metadata tab
3. Confirm that your Tale has the CC4 license assigned to it
4. Confirm that the Tale's Published Location reads This Tale has not been published
5. Note "Datasets Used: No citable data"
6. Add multiple dataset via External Datasets (Humans and Hydrology, Replication Data for Direct Democracy...)
7. Navigate to Run > Metadata
8. Confirm that there are citations for the external datasets
9. Click the "Edit" button to edit the metadata fields
10. Confirm editable fields (Title, Authors, Category, Environment, License, Description, Illustration)
11. Valid environments should be listed in the Environment dropdown menu
12. Valid licenses should be listed in the License dropdown (CC4 and CC0)
13. Change the name of the Tale, Category, Description, Illustration, Public, License, and add multiple authors
14. Select Save. Confirm that you see a notification telling you that the Tale was saved.
15. Confirm that changes are reflected in the page.
16. Refresh the page. Your changes should be preserved between page refreshes, confirming that they are being properly persisted.
17. Confirm that the new Tale authors are displayed on the Tale card

• Metadata tab - Non-Owned Tale (e.g., Public LIGO Tale)

1. Open a Tale that you don't own
2. Navigate to the metadata page
3. Note that there isn't an 'Edit' button present
4. Confirm read-only fields (Title, Created By, Authors, Category, Environment, License, Description, Datasets Used, Related Identifiers, Publised Location, Date Created, Last Updated)

• Home

1. View a tale that you own
2. Select Files > Home. Confirm that your home directory displays
3. Create a folder and upload a file to the folder
4. Confirm file operations: Copy, rename, remove, download, Move To...
   1. Copy file/folder
   2. Rename file/folder
   3. Download file/folder
   4. Upload file / create folder with duplicate name

Issue: File context menu no longer works after upload file but works during upload. Context menu doesn't work after after rename. whole-tale/ngx-dashboard#259

• External data

1. Precondition: Register one or more Datasets (e.g. doi:10.18739/A29G5GD0V)
   1. Confirm notification progress and success
2. Create a new tale
3. Confirm "External Data" list is initially empty
4. Confirm (+) button ("Select Data modal") behavior
   1. Confirm that "WT Data Catalog" shows all registered datasets
   2. Confirm that "My External Data" filters to only show your registered datasets
   3. Select a dataset
   4. Confirm that clicking "Add Selected" adds the dataset to the "Selected Data" column
   5. Confirm that "Remove Selected" data to dissociate it with a Tale
   6. Using the dataset above, add the dataset to the Tale, by clicking "Save"
   7. Confirm that the Tale's "External Data" list matches what was saved
   8. Remove the dataset from the Tale
   9. Confirm "External Data" list matches what was saved
   10. Select a dataset but then click out of the modal or click "Cancel"
   11. Confirm that the changes weren't saved

• Tale Workspace

1. Confirm create folder
2. Confirm upload file
3. Select "+" button has options "New folder", "Upload file", "Select Data..."
4. Folder operations
   1. Confirm context menu Copy, Rename, Download, Remove
   2. Confirm rename folder
   3. Confirm download folder (this should result in a zip file)
   4. Confirm remove folder
5. File operations
   1. Upload a file
   2. Confirm context menu Copy, Rename, Download, Remove
   3. Confirm rename file
   4. Confirm download file
   5. Confirm remove file
6. Select "Select Data..."
   1. Note vertical navigation with Tale Workspaces
   2. Copy to Workspace from another workspace
   3. Move to workspace from another workspace

Issue: UI does not refresh after move or copy to workspace, but file is copied.
Note: Why can't I see files from public tales (e.g., LIGO)? What constitutes an accessible tale?

• Files - Non-Owned Tale

1. Open a Tale that you don't have write permissions to
2. Attempt to add data to the workspace and external data
3. Note that the "+" icons are greyed out, preventing you from doing so
4. Note that Home directory isn't present

• Export Tale

1. Open your copy of the LIGO Tale or import https://girder.test.wholetale.org/api/v1/integration/zenodo?doi=10.5281%2Fzenodo.490449&resource_server=sandbox.zenodo.org
2. Select "Export Tale" from Tale menu
3. Confirm that a download starts for the ZIP archive
4. Open the archive
5. Confirm that exported ZIP is a valid Bag
   1. Must contain bagit.txt, bag-info.txt
   2. Optionally pip install bdbag and bdbag --validate full .
6. Run the exported tale and confirm that it builds and runs as expected
   1. Follow instructions in README.md

Test case: Need to fix bdbag validation steps.
Issue: Cannot export public tale -- "message": "Write access denied for tale 59f0b91584b7920001b46f2e (user 59fb6165f7e8790001da4e8b)." Disable menu as we do for publish whole-tale/ngx-dashboard#260

Versions

• Tale Versioning

1. Create a new tale or open a tale that you own
2. Expand Tale Version panel (on the right)
3. Create a New Version: init
4. Modify metadata, upload a file, and attach a dataset to the Tale
5. Create another new version: modified
6. While viewing Run > Files > External Data, restore from init - you should see no datasets
7. Rename the modified Version to something else
8. In the same view, restore back to something else - you should see the dataset you added
9. Run the tale. Confirm the versions are available under ../versions in-container and that the contents including metadata are as expected
10. View Info for a version
11. Rename a version. Confirm rename in container.
12. Export a version. Confirm exported version matches selected version.
13. Delete a version. Confirm deletion in container.

Note: I always force new version since it's checked. When wouldn't I? (Doc issue or hide checkmark)

• Publish/export version

1. Create a new tale
2. Upload files but do not create a version
3. Publish the tale
4. Note that there is a new version in the version panel
5. Confirm that the data files in the published artifact are correct
6. Repeat for export
7. Confirm manifest contents including vocabulary

Recorded Runs

• Create tale from Github

1. Create tale from Github repo https://github.com/whole-tale/rrun-test
2. Select JupyterLab environment
3. Create and start the tale
4. Select "..." > "Recorded Run" with script run.sh
5. Confirm recorded run completes successfully
6. Select history icon > "Perform Run" with script notfound.sh
7. Confirm recorded run fails
8. Select Files > Recorded Runs. You should see two runs, one COMPLETED and one FAILED
9. Select Files > Saved Versions. You should see a single version. Confirm that the version name matches the history panel.
10. Rename the version, confirm that it updates in the File manager and history panel for runs.
11. Attempt to remove the version, confirm error "ersion is in use by a run and cannot be deleted."
12. Rename one of the runs. Confirm it is renamed in the history panel and in-container.
13. Remove both of the runs. Confirm they are removed in the history panel and in-container.
14. Confirm that you can now remove the version

• Copy versions and runs

1. Using the above tale, select "..." > "Recorded Run" with script run.sh
2. Share the tale with a user with view-only permissions
3. As the view-only, run the tale and confirm versions and runs are copied
4. Remove copy of the tale
5. Share the tale with edit permissions
6. Confirm that versions and runs can be renamed but not deleted.
7. Share the tale with owner permissions
8. Confirm that versions and runs can be deleted

Settings

• Default State

1. Navigate to the Settings page
2. Confirm there are empty settings for DataONE, Dataverse, and Zenodo

• Connect to Zenodo

1. Click "Connect Account" in the Zenodo section
2. Confirm that the connection modal opens
3. Confirm that sandbox.zenodo.org and zenodo.org are listed
4. Select sandbox.zenodo.org from repository list
5. Select "Get from Zenodo"
   1. Login to Zenodo if necessary
   2. Create token
   3. Copy token
6. Enter your API key into WT modal
7. Click "Connect"
8. Confirm that you see "Authorized on "
9. Select disconnect, confirmation window should appear
10. Repeat test with invalid key, confirm error message appears
11. Repeat for each Zenodo server
12. Confirm "Get from" link matches selected repo

• Connect to Dataverse

1. Click "Connect Account" in the Dataverse section
2. Confirm that the connection modal opens
3. Enter your API key for one of the repositories
4. Click "Connect"
5. Confirm that you see "Authorized on "
6. Select disconnect, confirmation window should appear
7. Repeat test with invalid key, confirm error message appears
8. Repeat for each Dataverse server
9. Confirm "Get from" link matches selected repo

Issue: Need to remove dev2, which no longer exists.

• Connect to DataONE

1. Click "Connect Account" under the DataONE repository
2. Confirm that the connection modal opens
3. Enter your JWT for one of the repositories
4. Confirm that you see "Authorized on "
5. Select disconnect, confirmation window should appear
6. Repeat test with invalid key, confirm error message appears
7. Repeat for each DataONE server

Issue: Need to change CN to cn-sandbox.test.dataone.org (migration?). cn-stage-2.test.dataone.org/cn/v2 is redirecting. whole-tale/girder_wholetale#528

• Confirm tokens retained across logins
  #370

1. Connect one or more accounts as outlined above
2. Logout and login again
3. Using Swagger UI, hit the /user/me endpoint to confirm tokens are still present

Tale Creation

• Create JupyterLab Tale

1. Select the Create New Tale button on the 'Browse' page
2. Enter title "Test Tale"
3. Select JuptyerLab environment
4. Select "Create New Tale"
5. Confirm that you're brought to the 'Metadata' page and the "Run Tale" button is visible
6. Click "Run Tale"
7. Confirm notification dialog opens and progress order is correct
8. Confirm that you are redirected to "Interact" tab with Tale launching spinner
9. During build image step, select View Logs. Confirm logs are accessible.
10. Wait for launch to complete
11. Select "Return to Dashboard"
12. Confirm the Tale is shown under "Currently running"
13. Select "View", confirm that the interact tab is displayed with JupyterLab IDE
14. Select popout icon and confirm that JupyterLab IDE displays
15. Stop and delete the tale

• Create RStudio Tale

1. Create tale from Git repo https://github.com/whole-tale/rstudio-example/
2. Select environment RStudio 4.0.2
3. Run tale and confirm RStudio IDE displays in iframe and works in popout
4. Open Code/1_credit_permissiveness.R
5. Confirm outputs are created in Outputs without error

Issue: Migration required for R 4.0.2 due to old WT_RSTUDIO_* env vars.
Issue: Migration step to add environments

• Compose Jupyter Notebook Tale

1. Create tale from Git repo https://github.com/whole-tale/jupyter-example/
2. Select environment Jupyter Notebook
3. Run tale and confirm Jupyter Classic IDE displays in iframe and works in popout
4. Add usco2005.xlsx from Humans and Hydrology dataset
5. Run notebook, confirm it produces most glorious map of Texas

• Compose JupyterLab Tale

1. Create tale from Git repo https://github.com/whole-tale/jupyter-example/
2. Select JupyterLab environment
3. Run tale and confirm JupyterLab IDE displays in iframe and works in popout
4. Confirm JupyterLab IDE displays
5. Add usco2005.xlsx from Humans and Hydrology dataset
6. Run notebook, confirm it produces most glorious map of Texas

• Compose JupyterLab with Spark

1. Create tale from Git repo https://github.com/whole-tale/spark-example/
2. Select Jupyter with Spark environment
3. Run tale and confirm JupyterLab IDE displays in iframe and works in popout
4. Run notebook and confirm version is 2.4.3

Issue: Repeated by different build errors, latest appears to be due to memory limits (I deployed too small of test instances?)

#42 260.3 Solving environment: ...working... Killed
#42 ERROR: executor failed running [/bin/sh -c conda install --quiet -p ${NB_PYTHON_PREFIX} -y 'pyarrow' &&     conda clean --all -f -y]: runc did not terminate sucessfully
------
> [35/39] RUN conda install --quiet -p /srv/conda/envs/notebook -y 'pyarrow' &&     conda clean --all -f -y:
------
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = failed to build LLB: executor failed running [/bin/sh -c conda install --quiet -p ${NB_PYTHON_PREFIX} -y 'pyarrow' &&     conda clean --all -f -y]: runc did not terminate sucessfully

• Compose MATLAB Desktop Tale

1. Create tale from Git repo https://github.com/whole-tale/matlab-example/
2. Select MATLAB Desktop environment
3. Run tale and confirm MATLAB Web Desktop IDE displays in iframe and works in popout
4. Open and run multiplicative_arima_example_script.m, confirm outputs
5. Open and run multiplicative_arima_example_livescript.m, confirm outputs

Issue: Build error during install of toolboxes under R2020b. Investigating.

• Compose MATLAB Jupyter Kernel Tale

1. Create tale from Git repo https://github.com/whole-tale/matlab-example/
2. Select MATLAB Jupyter Kernel environment
3. Run tale and confirm JupyterLab IDE displays in iframe and works in popout
4. Open and jun multiplicative_arima_example.ipynb, confirm outputs match multiplicative_arima_example.html

• Compose MATLAB Linux Desktop Tale

1. (Only test if present)
2. Create tale from Git repo https://github.com/whole-tale/matlab-example/
3. Run tale and confirm Xpra-based MATLAB IDE displays in iframe and works in popout
4. Open and run multiplicative_arima_example_script.m, confirm outputs

• Compose STATA Desktop Tale

1. Create tale from Git repo https://github.com/whole-tale/stata-example/
2. Run tale and confirm Xpra-based STATA IDE displays in iframe and works in popout
3. Confirm desktop icons work (Firefox, STATA, terminal)
4. Minimize and maximize windows from task bar
5. Run STATA, run example.do, confirm outputs

• Compose STATA Jupyter Kernel Tale

1. Create tale from Git repo https://github.com/whole-tale/stata-example/
2. Run tale and confirm JupyterLab IDE displays in iframe and works in popout
3. Run example.ipynb, confirm outputs match example.html

• Compose OpenRefine Tale

1. Create a new tale based on the OpenRefine environment
2. Run tale and confirm OpenRefine IDE displays in iframe and works in popout

Issue: Can we just remove OpenRefine? It's a waste of time to even test. How many openrefine Tales are there?

• Too many instances

1. Run 2 tales
2. Attempt to run another Tale
3. Confirm that you receive an error that you have the max limit

Analyze in Whole Tale

These test cases cover potential situations that can occur when importing datasets from Dataverse.

• Import dataset from Dataverse

1. Remove all running Tale instances
2. Navigate to https://girder.test.wholetale.org/api/v1/integration/dataverse?datasetPid=doi%3A10.7910%2FDVN%2F3MJ7IR&siteUrl=https%3A%2F%2Fdataverse.harvard.edu
3. Confirm that the Tale title reads as Replication Data for: "Agricultural Fires and Health at Birth"
4. Confirm that the only item in the Input data section matches the uri with Data Source appended
5. Confirm that no environment is selected
6. Confirm that the Create New Tale button is disabled
7. Select an environment
8. Confirm that READ/WRITE is selected
9. Click Create New Tale
10. Confirm registration notification progress and success
11. Confirm that you are redirected to the Metadata page and "Run Tale" is enabled
12. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
13. Confirm that the data exists under Tale Workspace
14. Confirm that the Tale's category is science

Issue? This test case only works if you don't have a tale with the dataset registered already whole-tale/girder_wholetale#529
Test case: Category is social science

• Import dataset from DataONE: READ-ONLY

1. Remove all running Tale instances
2. Navigate to https://girder.test.wholetale.org/api/v1/integration/dataone?uri=https%3A%2F%2Fsearch.dataone.org%2Fview%2Fdoi%3A10.18739%2FA2VQ2S94D&title=Fire%20influences%20on%20forest%20recovery%20and%20associated%20climate%20feedbacks%20in%20Siberian%20Larch%20Forests%2C%20Russia&environment=RStudio
3. Confirm that the Tale title reads as Fire influences on forest recovery and associated climate feedbacks in Siberian Larch Forests, Russia
4. Confirm that the only item in the Selected data section matches the uri with Data Source appended
5. Confirm that Rstudio is selected in the Environments widget
6. Confirm that the Create New Tale button is enabled
7. Confirm READ ONLY is selected
8. Click Create New Tale
9. Confirm that you are redirected to the Metadata tab
10. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
11. Confirm that the data exists in the Tale under External Data
12. Confirm that the Tale's category is science
13. Confirm that the Datasets Used and Related Identifiers are updated to reflect the dataset

Issue: Test case only works if environment matches exact name. This has alway been fragile.

• Import from DataONE: READ-WRITE

1. Navigate to https://girder.test.wholetale.org/api/v1/integration/dataone?uri=https%3A%2F%2Fsearch.dataone.org%2Fview%2Fdoi%3A10.18739%2FA2VQ2S94D&title=Fire%20influences%20on%20forest%20recovery%20and%20associated%20climate%20feedbacks%20in%20Siberian%20Larch%20Forests%2C%20Russia&environment=RStudio
2. Confirm that the Tale title matches the dataset
3. Confirm that the only item in the Selected data section matches the uri with Data Source appended
4. Confirm that the Create New Tale button is disabled
5. Select READ/WRITE
6. Click Create New Tale
7. Confirm that the notification bar appears & properly updates
8. Confirm that you are redirected to the run page
9. Confirm that the Tale name matches the Tale Name in the Create Tale Modal
10. Confirm that the data exists in the Tale Workspace

Issue?: During register, dataset appears as subfolder in workspace, but is eventually removed
Issue: Import fails with I/O error whole-tale/girder_wholetale#530

• Import from DataONE: alternate sites
  1. "proveit": https://dashboard.test.wholetale.org/browse?api=https%3A%2F%2Fcn-sandbox.test.dataone.org%2Fcn%2Fv2&name=Dataone%20Dataset&uri=https://dev.nceas.ucsb.edu/view/doi:10.5072/FK2K075M25
  2. "Alpha Helix 2001...": https://dashboard.test.wholetale.org/browse?api=http%3A%2F%2Fcn-stage.test.dataone.org%2Fcn%2Fv2&name=Dataone%20Dataset&uri=https://test.arcticdata.io/view/doi:10.5065/D6N014NC
  3. Remove all running Tale instances
  4. Navigate to each of the two links above
  5. Select an environment
  6. Click Create New Tale
  7. Confirm that the correct data exists in the Tale under External Data

Test case: Update CN
Issue: proveit dataset registers as a single file "FK2K075M25". Need to investigate (Girder logs WARNING: No object was found in the index for doi:10.5072/FK2K075M25.)

• Import from Zipfile
  1. Export a tale you own or copy of LIGO tale
  2. Import via API

export GIRDER_TOKEN=<your token>
curl -X POST -H "Content-Type: application/zip" --data-binary "@<path-to-zip>.zip" --header "Girder-Token: ${GIRDER_TOKEN}" "https://girder.test.wholetale.org/api/v1/tale/import"

1. Confirm tale imports, builds and run
2. Try with invalid token: (You must be logged in)
3. Try with invalid zipfile (Bad bag)

Tale metadata tests

The purpose of these tests are to confirm that the metadata files (manifest.json, environment.json, LICENSE) we generate are correct.

• manifest.json

1. Launch a Tale that you own
2. Add a dataset to the Tale
3. Add at least one author
4. Add at least one file to the workspace
5. Create a version
6. Export the Tale (as BagIt)
7. Open the manifest.json file under metadata/
8. Confirm wt context present
9. Confirm @id references the correct Tale ID
10. Confirm that the author is listed under schema:author
11. Confirm that you are listed under createdBy
12. Confirm that the file(s) in the workspace are properly listed in aggregates
13. Confirm that the external dataset is referenced in aggregates
14. Confirm that the dataset is also listed under wt:usesDataset
15. Confirm dct:hasVersion is present and matches your created version
16. Optionally confirm rdflib can parse:

from rdflib import Graph

import urllib.parse
urllib.parse.uses_relative.append('arcp')
urllib.parse.uses_netloc.append('arcp')

g = Graph().parse(source="manifest.json", format="json-ld")
print(g.serialize(format='turtle', indent=2).decode())

• environment.json

1. Open the archive from the manifest.json test
2. Navigate to metadata/environment.json
3. Open the girder web API
4. Locate and GET the Image that the Tale used
5. Confirm that the config section from girder matches the config section in environment.json

• LICENSE

1. Open the archive from the manifest.json test
2. Open the LICENSE file
3. Navigate to Run > metadata
4. Confirm that the opened LICENSE file matches the selected license on the metadata page
5. Change the Tale's license
6. Export the Tale
7. Confirm that the LICENSE changed appropriately

Tale Sharing Tests

These tests are centered around the feature for sharing Tales with other Whole Tale
users. To run these tests, register at least one other account, referred to as User B.

• Case 1: Sharing with 'Can View' permissions

  1. Create a new Tale
  2. Add data to the workspace and add at least one dataset
  3. Navigate to the 'Share' tab
  4. Share the Tale with User B
  5. Confirm that the default permission is 'Can View'
  6. Switch the account to User B
  7. Navigate to the 'Shared with Me' tab on the Browse page
  8. Confirm that you see the Tale from step 1
  9. Confirm that you can't edit the Workspace, External Data, or metadata
  10. Unshare tale with User B
  11. Note modal displays that tale has been unshared.

• Case 2: Sharing with 'Can Edit' permissions

  1. Create a new Tale
  2. Add data to the workspace and add at least one dataset
  3. Navigate to the 'Share' tab
  4. Share the Tale with User B
  5. Set the permissions to 'Can Edit'
  6. Switch the account to User B
  7. Navigate to the 'Shared with Me' tab on the Browse page
  8. Confirm that you see the Tale from step 1
  9. Remove one of the files from the Workspace
  10. Add a dataset to External Data
  11. Change a field in the metadata (title, description, etc)
  12. Switch to User A
  13. Confirm that the changes made to the Tale persisted

• Case 3: Edit sync modal

  1. Login as User A and User B in separate browser sessions
  2. As User B, select "Shared with me"
  3. As User A, share a tale with User B; as User B confirm the tale appears
  4. As User B, select "View" on the shared tale
  5. As User A and B edit the tale metadata
  6. As User A change title, author, category, description; save changes
  7. As User B confirm Tale edited modal, select Yes, confirm updates to metadata

Publishing tests

• Case 1: No Accounts Configured

  1. Disconnect all of your external accounts
  2. Open a Tale that you own
  3. Click the ellipsis dropdown on the Run page
  4. Click Publish tale...
  5. Confirm that your are notified to link accounts on the settings page
  6. Click Go to Settings
  7. Confirm you are re-directed to settings

• Case 2: Publishing to DataONE

  1. Connect to DataONE (dev server)
  2. Open a Tale that you own
  3. Add some data to it
  4. Click Publish tale...
  5. Confirm that the publishing modal opens
  6. Confirm that the third party that you are connected to appears in the dropdown
  7. Select the third party
  8. Click Publish
  9. Once complete, navigate to the metadata page
  10. Confirm that the published location is present
  11. Visit the published location
  12. Confirm that you see the following files plus yours
      1. metadata.xml
      2. LICENSE
      3. README.md
      4. manifest.json
      5. environment.json
  13. After publishing to DataONE, use girder to query the Tale
  14. Ensure that the 'pid' field in the 'publishInfo' object has an identifier in a doi format

Issue: Can't even select dev server to publish to.

• Case 3: Re-publish Tale to DataONE

1. Launch a Tale that has been published before
2. Take note of the Published Location in Run > metadata
3. Re-publish the Tale to the third party
4. Confirm that the Published Location has changed to the correct package landing page

Zenodo integration tests

• Register Zenodo data

1. On a new or existing tale, select "Files" > "External Data"
2. Select "+" > "Web (DOI or URL)"
3. Search for Concept DOI doi:10.5281/zenodo.16384
4. Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_1
5. Search for URL https://zenodo.org/record/1172960#.XidIvlNKh24
6. Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_2
7. Search for DOI doi:10.5281/zenodo.1172960
8. Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_2
9. Search for Concept DOI doi:10.5281/zenodo.608984
10. Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_2
11. Select Register
12. Select "+" > "WT Data Catalog"
13. Add "Belmot Forum..." dataset to Tale
14. Run Tale and confirm contents on file system match Zenodo

The register tests the following cases.

• Datasets with multiple files

• Datasets with multiple versions

• Concept DOI

• Zenodo dataset via AiWT

1. Open https://dashboard.test.wholetale.org/browse?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.820575&name=Automotive%20Sensor%20Data
2. Confirm Source Data URL and Title match above URL and "READ ONLY" is selected
3. Select JupyterLab environment
4. Select 'Create New Tale'
5. Confirm dataset is mounted read-only, citation and related ids are as expected,
6. Run IDE and use terminal to ls ../data/Auto* to confirm zipfile is not extracted

Test case: Could setup test tale with valid MD5 check, etc.

The register tests the following cases.

• Dataset with single file (zipfile)

• Zenodo Binder via AiWT

1. Open https://dashboard.test.wholetale.org/browse?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.3242073&environment=JupyterLab&name=Simple%20requirements.txt%20based%20example%20for%20repo2docker&asTale=true
2. Confirm Source Data and Title match URL and "READ WRITE" is selected
3. Select 'Create Tale'
4. Confirm image builds and notebook can run

Note: Image build starts with step 22/install-miniforge? But why?
Issue: Intermittent 503 from https://keyserver.ubuntu.com/. Need to retest.

• Zenodo Tale via AiWT

1. Open https://girder.test.wholetale.org/api/v1/integration/zenodo?doi=10.5281%2Fzenodo.490449&resource_server=sandbox.zenodo.org
2. Confirm Water Tale is created.
3. Confirm citation on Metadata view
4. Confirm DOI matches above
5. Use the Swagger UI to confirm the publishInfo field on the tale object contains pid, repository, repository_id, uri
6. Run the tale and and notebook, confirm it works as expected (produces glorious map of Texas)

• Publish/import to/from Zenodo Sandbox

1. Follow instructions above to configure your API key for Zenodo sandbox
2. Create your own Tale and publish to sandbox.zenodo.org
3. Confirm publishInfo contains Sandbox entry
4. Delete tale
5. Import tale from Zenodo
6. Confirm publishInfo
7. Modify the tale
8. Re-publish to sandbox.zenodo.org
9. Delete tale
10. Import tale from Zenodo
11. Confirm publishInfo
12. Download and extract the exported Tale
13. Execute the run-local.sh command to run the Tale locally

Git Integration Tests

• Compose Tale from Git Repository
  Covered by other tests

1. Select the Create New Tale button on the 'Browse' page
2. Select the Create Tale from Git Repository option
3. Paste a link to a Git repository in the correct field
4. Add a name and pick a compute environment
5. Create the Tale
6. Confirm notification progress and success
7. Confirm that the git repository was cloned to the workspace/ root
8. Run the tale
9. Confirm interaction via git tools (may require environment customization)

• Connect to Git

1. Create a new empty tale
2. Select "Connect to Git Repository" menu item from Tale menu
3. Confirm notification progress and success
4. Once complete, navigate to the Tale workspace and confirm that the repository has been cloned

Other tests

• Citations

1. Add files from multiple external datasets (e.g.: doi:10.5281/zenodo.3381658, doi:10.5065/D6862DM8, doi:10.5063/F10K26VR)
2. Confirm Datasets Used citations and related identifiers are updated

Event Notification Tests

• Tale sharing/unsharing

1. Login as user1 and user2 in separate browser sessions
2. As user2, select "Shared with me"
3. As user1, share a tale with user2; as user2 confirm the tale appears
4. As user2, select "View" on the shared tale
5. As user1 and 2 edit the tale metadata
6. As user1 save changes; as user2 confirm Tale edited modal
7. As user1, unshare the tale; as user2 confirm Tale unshared modal
8. As user1, re-share with user2
9. As user2, select "Shared with Me"
10. As user1, delete the shared tale; as user2 confirm that the tale disappears

• Import tale

1. Import a tale from Github, Zenodo, and as Binder
2. Confirm metadata updates accordingly after import completes

Vocabulary Tests

• Vocabularies

1. Go to https://vocabularies.test.wholetale.org/wt/1.0/index.html
2. Confirm that it matches https://whole-tale.github.io/serialization-format/wt/1.0/

Issue: Oops! Need to update vocab for 1.1... PR needs to be merged?

Forward Authentication Test

• Forward authentication

1. Start a tale
2. Confirm that the tale is accessible via iframe and popout
3. Right click popout icon and copy URL (this is necessary to get JupyterLab token)
4. Open an incognito window and paste the copied URL
5. Confirm that you are prompted for authentication and can access the tale once authenciated
6. Login as a user who does not have permission to access the tale
7. Confirm that they receive an access denied errror

Changing the Environment

• Changing the environment

1. Create a new JupyterLab tale
2. Run the Tale, confirm JupyterLab IDE displays
3. Edit metadata, change environment to "RStudio"
4. Confirm icon changes in title bar and Environment section
5. Stop/run tale or rebuild/restart and confirm RStudio IDE displays

Idle Instances

• Idle instance culling

1. Create and start a MATLAB tale and a JupyterLab tale
2. Confirm both instances start and IDEs are accessible
3. Close all IDE tabs/windows
4. Wait for 2 hours and 24 hours respectively
5. Confirm both instances are stopped automatically

[craig-willis]

Identified issues:

• Run local:
  • Globus transfer fails because globus-sdk is missing from the image and keychain.json isn't configured. The user also has no idea this needs to happen (Document Globus configuration for run-local whole-tale#115
  • Error on DERIVA zip (BDBag fetch support for DERIVA zips whole-tale#114)
• Splash page: TOS points https://dashboard.test.wholetale.org/undefined/tos. Dashboard is still using hardcoded girder.local.wholetale.org ngx-dashboard#255.
• Login: ORCID login fails with 500 error -- on test and dev, but not stage. Issue with Client ID not respecting requests from test.wholetale.org? Login is working on stage. (500 on AiWT When Authenticating with ORCID & Globus girder_wholetale#459)
• AiWT URL encoding still not right when user not logged in: AiWT URL encoding issue remains ngx-dashboard#258.
• File Manager
  • File context menu no longer works after upload file but works during upload. Context menu doesn't work after after rename. Context menu no longer works after upload/rename ngx-dashboard#259
  • UI does not refresh after move or copy to workspace, but file is copied.
• Export: Cannot export public tale. Disable menu as we do for publish Disable export for users without write permission ngx-dashboard#260
• DataONE: Need to change CN Change staging cn to cn-stage girder_wholetale#528
• Migration:
  • Required for R 4.0.2 due to old WT_RSTUDIO_* env vars
  • Need to add all environments to prod too
• Jupyter/Spark: Repeated but different build errors, possibly due to small test VMs. Also had intermittent 503 from https://keyserver.ubuntu.com/.
• Matlab: Build error during install of toolboxes, possibly due to small test VMs.
• DataONE Import fails with I/O error Importing D1 dataset fails with I/O error girder_wholetale#530
• DataONE publish: Can't even select dev server to publish to.
• Import dataset from Dataverse: Test case only works if you don't have a tale with the dataset registered already AiWT matches any existing tale with selected dataset girder_wholetale#529

Test case issues:

• Update D1 CN (stage-2 -> stage)
• Remove proveit case, since we don't allow registering dev datasets? Oroveit dataset registers as a single file "FK2K075M25". Need to investigate (Girder logs WARNING: No object was found in the index for doi:10.5072/FK2K075M25.)
• Zenodo dataset via AiWT (and others) -- Could setup test tale with valid MD5 check, etc.
• Fix bdbag validation steps.
• Test cases only work if environment matches exact name. This has alway been fragile.

Notes:

• Should we acknowledge Cornell, UNC, UMich?
• Tale export/import doesn't retain permissions (e.g., execute bit on run.sh)
• Why can't I see files from public tales (e.g., LIGO)? What constitutes an accessible tale?
• I always force new version since it's checked. When wouldn't I? (Doc issue or hide checkmark)

[craig-willis]

On to RC2...