v1.2rc1 testing (Chrome/MacOS) #171

craig-willis · 2022-09-19T15:49:06Z

Test plan walkthrough on Staging

Test Plan

https://github.com/whole-tale/wt-design-docs/issues/new?template=TEST_PLAN.md

Note: For all tests, repeat for supported browser/OS combinations.

Preconditions:

Clear browsing history or open incognito/private session
Disconnect all third party integrations

End-to-End Test

End to end test

Open private session as user1
https://girder.stage.wholetale.org/api/v1/integration/zenodo?doi=10.5072%2Fzenodo.1021503&resource_server=sandbox.zenodo.org
# when prompted
Start the interactive environment (Run Tale)
sh run.sh, confirm no errors
Start a recorded run using run.sh, confirm SUCCESS
Export tale
Run locally
Execute run.sh, confirm success
Share the tale view-only with user2
As user2, copy-on-launch. Confirm versions and runs were copied
Start a recorded run using run.sh, confirm SUCCESS
Export the tale
sh run-local.sh and access the interactive environment
In the interactive environment sh run.sh, confirm no errors

Notes:

Integration link fails if login via ORCID due to state parameter limit
DERIVA and Globus fetch fails on run local (known)

Unauthenticated access

Unauthenticated access

Open a new private/incognito window
Goto https://dashboard.stage.wholetale.org
Confirm that you are able to access the dashboard without signing in
Confirm only "Public Tales" display
Confirm "#" button appears in upper right
Select "View" on a public tale (e.g., LIGO)
Confirm that you are able to view metadata; browse and download files in workspace, external data, versions, and runs.
Confirm that tale menu displays only docs link.
Confirm that "Run Tale" button is disabled and that a tooltip displays "You must log in to run this tale"
Confirm that tale history panel displays versions and runs read only
Confirm navigation links are functional: Whole Tale logo, Tale Dashboard, Docs, File and Issue, Return to Dashboard

Signing in

Basic login flow

As unauthenticted user, go to https://dashboard.stage.wholetale.org
Note that you are redirected to /public
Select "#" and complete auth flow
Confirm that you are taken to the /public panel as a logged in user. All features for signed-in users should be functional (My Tales, Shared Tales, Create Tale, etc).
Logout, confirm "#" button appears and only "Public Tales" is accessible
Open a public tale (e.g., LIGO)
Select "#", confirm that you are accessing the tale as a signed-in user
Logout, confirm that you are accessing the tale as an unauthenticated user

Confirm auth providers

Select "#" button
Select "Access Whole Tale" button. You should be prompted to login via Globus Auth.
Confirm institutional login
Confirm Google login
Confirm ORCID login

Basic logout flow

As an authenticated user, select the "User" menu
Select the "Log out" menu item. You should be taken to the Public Tales catalog.
Select "#"
Note that you are not prompted to login to Globus again
Select User menu > Logout.
Go to https://app.globus.org/logout to logout from Globus
Go to https://dashboard.stage.wholetale.org
Select "#", you should be promted to login

Return-route for non-logged in users

Open private window
Attempt to access https://dashboard.stage.wholetale.org/mine
You should be redirected to Globus auth
Enter your credentials
You should be redirected to the "My Tales" page.

Forward auth

Create and start an empty tale instance
Copy the link
Start a new private session
Attempt to access the link
Login as user with permission on the tale. Confirm that the IDE displays.
Login as user without permission. Confirm instance access is denied.

Analyze in WT (unauthenticated user)

Open new private session
Go to https://dashboard.stage.wholetale.org/mine?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.820575&name=Automotive%20Sensor%20Data
You should prompted to #
Confirm "Create new Tale" dialog opens with title "Automotive Sensor Data" and datasource populated

Navigation

General navigation

Go to https://dashboard.stage.wholetale.org and #
"Public Tales" should display (since you were on it when you signed in)
Click "Create New Tale" > "Create New Tale". The Create Tale modal should display. Click "Cancel" to close the modal
Select the "i" icon at the upper right corner; it should open the User Guide in new tab. Hovering should display tooltip.
Select the "Notification" button next to the "i" icon; it should display the notification panel. Hovering should display tooltip.
Select the "bug" icon; it should open https://github.com/whole-tale/whole-tale/issues in new tab (after authentication). Hovering should display tooltip.
Select the "?" icon; it should display the "About" page with TOS and acknowledgements.
Select "Tale Dashboard" link. "My Tales" page should display
Mouse over a Tale and select "View". The Run page should display
Click the "user" icon at the top right

You should see a dropdown expand
The dropdown should contain the user's first/last name and gravatar image
The dropdown should provide a link to the Settings page
The dropdown should provide an option to Logout
Select the "Settings" option; it should bring you to the Settings page

Tale Dashboard

Preconditions:

Authenticated user
Assumes production Tales present (e.g., LIGO, Materials, etc).
No running instances
General

Select "My Tales" link. Confirm "My Tales" page displays
Confirm that no Tales are running
Select "Shared with Me" link. Confirm "Shared with Me" page displays,

Search

Select "Public Tales" link. Confirm "Public Tales" page displays
Enter "LIGO" as search term. Confirm only LIGO tale displays (by K. Kowalik)
Clear search term. Confirm all tales display

View tale

Select "View" button on LIGO Tale
Confirm that Metadata page displays by default
Confirm that you cannot delete or edit Tale properties
Select Back button. Confirm that you are taken back to Public Tales page
Select "View" button on Tale you own
1. Confirm that the Run and Close buttons are present
2. Confirm that you can edit Tale properties
3. Select "Close"
4. Confirm that you are taken back to "My Tales"

Launch instance

Select "Run Tale" button on "Ligo" tale
1. Confirm you are prompted to Copy and Run
2. Select "Confirm"
3. Confirm that you are redirected to the Tale's "Metadata" page
4. Confirm "Copy" label appears under title
5. Confirm that the "Edit" button is present
6. Select "Run Tale"
7. Confirm that the "Interact" page is displayed with spinner
8. Confirm that the notification dialog appears
9. Confirm that the step progression is correct
10. Confirm that the "View Logs" button shows the build logs in a modal
11. Navigate to the "My Tales" page
12. Confirm that the Tale is shown under "Currently Running"
13. Select "View" on the Tale card
14. Confirm that "Interact" page is displayed
Select "Stop" to end the Tale instance
1. Confirm that the Tale has stopped
2. Confirm that the button has changed to Run Tale
Navigate to "My Tales"
1. Confirm that there are no Tales under "Currently Running"
2. Select the "Run Tale" button on a Tale you own
3. After the instance is launched, navigate back to the My Tales page
4. Confirm that you cannot select the delete button on the My Tales page
5. Confirm that the "Run Tale" button has been replaced with "Stop Tale"
6. Confirm that the Tale is shown at the top of the page under "Currently running"

Managing Data

Preconditions

Empty home directory
No registered data
Register General

Open an owned Tale
Select 'External Data' > "+" > Web (DOI or URL)
Confirm list of providers matches list of supported providers (Globus, DataONE, Dataverse, Zenodo)

Register DataONE data

Open an owned Tale
Select 'External Data' > "+" > Web (DOI or URL)
Confirm modal displays
Search for DOI 10.5065/D6862DM8
Confirm "Humans and Hydrology" dataset is found on DataONE.
Select Register
Confirm dataset appears in the Data Catalog
Confirm additional URL formats:

Dataverse

Register dataset doi:10.7910/DVN/SZ9YXZ
Confirm "Replication Data for: Direct Democracy and Women's Political Engagement" from Dataverse

Globus/MDF

Confirm DOI resolution for http://dx.doi.org/doi:10.18126/M2301J results in https://petreldata.net/mdf/detail/pub_30_shahani_twinmediated_v1.2
Register dataset https://petreldata.net/mdf/detail/pub_30_shahani_twinmediated_v1.2

DERIVA

Register dataset https://pbcconsortium.s3.amazonaws.com/wholetale/5ad7cdf55b0d5007601015b7ff1ea8d6/2021-11-09_21.47.58/Dataset_1-882P.zip

Run Tale

Preconditions:

No running Tale instances
General

On "Public Tales", click "Run" on LIGO tale (Copy on Launch)
Select "Run Tale" to start the instance
Confirm you are redirected to the Run page
1. Confirm that the Run panel displays the Tale image, title, and author
2. Confirm that the Run page displays the Interact, Files, Metadata and Share tabs
3. Confirm that Interact tab is displayed by default
4. Confirm notification panel displays and progression is correct
Select the "..." menu.
1. Confirm that menu options exist for "Rebuild Tale", "Restart Tale", "Save Tale Version", "Recorded Run", "Duplicate Tale", "Publish Tale", "Export Tale", "Connect to Git Repository..."
2. Confirm that the "Read the docs" link displays https://wholetale.readthedocs.io/en/latest/users_guide/run.html in a new tab or window (stable is for production)
3. Confirm that the "View Fullscreen" link displays the Run panel in full screen

Interact tab

Select the "Interact" tab
Confirm Jupyter environment is displayed in iframe
Hover over popout icon, confirm tooltip displays.
Select "popout" icon. Confirm Jupyter is opened in new tab,

Files tab

Select "Files" tab
You should see a vertical navigation menu with two sections CURRENT and ARCHIVED.
Under CURRENT you should see three tabs: "Home", "External Data", and "Tale Workspace".
Under ARCHIVED you should see Saved Versions and Recorded Runs under ARCHIVED
Confirm Tale Workspace is selected by default

Metadata tab - Owned Tale

Open a Tale that you own (e.g., LIGO Tale copy)
Navigate to the Metadata tab
Confirm that your Tale has the CC4 license assigned to it
Confirm that the Tale's Published Location reads This Tale has not been published
Note "Datasets Used: No citable data"
Add multiple dataset via External Datasets (Humans and Hydrology, Replication Data for Direct Democracy...)
Navigate to Run > Metadata
Confirm that there are citations for the external datasets
Click the "Edit" button to edit the metadata fields
Confirm editable fields (Title, Authors, Category, Environment, License, Description, Illustration)
Valid environments should be listed in the Environment dropdown menu
Valid licenses should be listed in the License dropdown (CC4 and CC0)
Change the name of the Tale, Category, Description, Illustration, Public, License, and add multiple authors
Select Save. Confirm that you see a notification telling you that the Tale was saved.
Confirm that changes are reflected in the page.
Refresh the page. Your changes should be preserved between page refreshes, confirming that they are being properly persisted.
Confirm that the new Tale authors are displayed on the Tale card

Metadata tab - Non-Owned Tale (e.g., Public LIGO Tale)

Open a Tale that you don't own
Navigate to the metadata page
Note that there isn't an 'Edit' button present
Confirm read-only fields (Title, Created By, Authors, Category, Environment, License, Description, Datasets Used, Related Identifiers, Publised Location, Date Created, Last Updated)

Home

View a tale that you own
Select Files > Home. Confirm that your home directory displays
Create a folder and upload a file to the folder
Confirm file operations: Copy, rename, remove, download, Move To...
1. Copy file/folder
2. Rename file/folder
3. Download file/folder
4. Upload file / create folder with duplicate name
5. Upload folder

External data

Precondition: Register one or more Datasets (e.g. doi:10.18739/A29G5GD0V)
1. Confirm notification progress and success
Create a new tale
Confirm "External Data" list is initially empty
Confirm (+) button ("Select Data modal") behavior
1. Confirm that "WT Data Catalog" shows all registered datasets
2. Confirm that "My External Data" filters to only show your registered datasets
3. Select a dataset
4. Confirm that clicking "Add Selected" adds the dataset to the "Selected Data" column
5. Confirm that "Remove Selected" data to dissociate it with a Tale
6. Using the dataset above, add the dataset to the Tale, by clicking "Save"
7. Confirm that the Tale's "External Data" list matches what was saved
8. Remove the dataset from the Tale
9. Confirm "External Data" list matches what was saved
10. Select a dataset but then click out of the modal or click "Cancel"
11. Confirm that the changes weren't saved

Tale Workspace

Confirm create folder
Confirm upload file
Select "+" button has options "New folder", "Upload file", "Select Data..."
Folder operations
1. Confirm context menu Copy, Rename, Download, Remove
2. Confirm rename folder
3. Confirm download folder (this should result in a zip file)
4. Confirm remove folder
File operations
1. Upload a file
2. Confirm context menu Copy, Rename, Download, Remove
3. Confirm rename file
4. Confirm download file
5. Confirm remove file
Select "Select Data..."
1. Note vertical navigation with Tale Workspaces
2. Copy to Workspace from another workspace
3. Move to workspace from another workspace
Upload folder (TODO)

Files - Non-Owned Tale

Open a Tale that you don't have write permissions to
Attempt to add data to the workspace and external data
Note that the "+" icons are greyed out, preventing you from doing so
Note that Home directory isn't present

Export Tale

Import https://girder.stage.wholetale.org/api/v1/integration/zenodo?doi=10.5281%2Fzenodo.1106153&resource_server=sandbox.zenodo.org
Select "Export Tale" from Tale menu
Confirm that a download starts for the ZIP archive
Open the archive
Confirm that exported ZIP is a valid Bag
1. bdbag --resolve-fetch all .
2. bdbag --validate full .
Run the exported tale and confirm that it builds and runs as expected
1. Follow instructions in README.md

Versions

Tale Versioning

Create a new tale or open a tale that you own
Expand Tale Version panel (on the right)
Create a New Version: init
Modify metadata, upload a file, and attach a dataset to the Tale
Create another new version: modified
While viewing Run > Files > External Data, restore from init - you should see no datasets
Rename the modified Version to something else
In the same view, restore back to something else - you should see the dataset you added
Run the tale. Confirm the versions are available under ../versions in-container and that the contents including metadata are as expected
View Info for a version
Rename a version. Confirm rename in container.
Export a version. Confirm exported version matches selected version.
Delete a version. Confirm deletion in container.

Publish/export version

Create a new tale
Upload files but do not create a version
Publish the tale
Note that there is a new version in the version panel
Confirm that the data files in the published artifact are correct
Repeat for export
Confirm manifest contents including vocabulary

Recorded Runs

Create tale from Github

Create tale from Github repo https://github.com/whole-tale/rrun-test
Select JupyterLab environment
Create and start the tale
Select "..." > "Recorded Run" with script run.sh
Confirm recorded run completes successully
Select history icon > "Perform Run" with script notfound.sh
Confirm recorded run fails
Select Files > Recorded Runs. You should see two runs, one COMPLETED and one FAILED
Select Files > Saved Versions. You should see a single version. Confirm that the version name matches the history panel.
Rename the version, confirm that it updates in the File manager and history panel for runs.
Attempt to remove the version, confirm error "ersion is in use by a run and cannot be deleted."
Rename one of the runs. Confirm it is renamed in the history panel and in-container.
Remove both of the runs. Confirm they are removed in the history panel and in-container.
Confirm that you can now remove the version

Copy versions and runs

Using the above tale, select "..." > "Recorded Run" with script run.sh
Share the tale with a user with view-only permissions
As the view-only, run the tale and confirm versions and runs are copied
Remove copy of the tale
Share the tale with edit permissions
Confirm that versions and runs can be renamed but not deleted.
Share the tale with owner permissions
Confirm that versions and runs can be deleted

Settings

Default State

Navigate to the Settings page
Confirm there are empty settings for Zenodo, Dataverse, DataONE, and ICPSR

Connect to Zenodo

Click "Connect Account" in the Zenodo section
Confirm that the connection modal opens
Confirm that sandbox.zenodo.org and zenodo.org are listed
Select sandbox.zenodo.org from repository list
Select "Get from Zenodo"
1. Login to Zenodo if necessary
2. Create token
3. Copy token
Enter your API key into WT modal
Click "Connect"
Confirm that you see "Authorized on "
Select disconnect, confirmation window should appear
Repeat test with invalid key, confirm error message appears
Repeat for each Zenodo server
Confirm "Get from" link matches selected repo

Connect to Dataverse

Click "Connect Account" in the Dataverse section
Confirm that the connection modal opens
Enter your API key for one of the repositories
Click "Connect"
Confirm that you see "Authorized on "
Select disconnect, confirmation window should appear
Repeat test with invalid key, confirm error message appears
Repeat for each Dataverse server
Confirm "Get from" link matches selected repo

Connect to DataONE

Click "Connect Account" under the DataONE repository
Confirm that the connection modal opens
Enter your JWT for one of the repositories
Confirm that you see "Authorized on "
Select disconnect, confirmation window should appear
Repeat test with invalid key, confirm error message appears
Repeat for each DataONE server

Connect to ICPSR

Click "Connect Account" in the ICPSR section
Confirm that the connection modal opens
Enter your ICPSR password. Note, this requires that your WT and ICPSR accounts have the same email address.
Click "Connect"
Confirm that you see "Authorized on "
Select disconnect, confirmation window should appear
Repeat test with invalid key, confirm error message appears

Confirm tokens retained across logins
#370

Connect one or more accounts as outlined above
Logout and login again
Using Swagger UI, hit the /user/me endpoint to confirm tokens are still present

Tale Creation

Create JupyterLab Tale

Select the Create New Tale button on the 'Browse' page
Enter title "Test Tale"
Select JuptyerLab environment
Select "Create New Tale"
Confirm that you're brought to the 'Metadata' page and the "Run Tale" button is visible
Click "Run Tale"
Confirm notification dialog opens and progress order is correct
Confirm that you are redirected to "Interact" tab with Tale launching spinner
During build image step, select View Logs. Confirm logs are accessible.
Wait for launch to complete
Select "Return to Dashboard"
Confirm the Tale is shown under "Currently running"
Select "View", confirm that the interact tab is displayed with JupyterLab IDE
Select popout icon and confirm that JupyterLab IDE displays
Stop and delete the tale

Create RStudio Tale

Create tale from Git repo https://github.com/whole-tale/rstudio-example/
Select environment RStudio 4.0.2
Run tale and confirm RStudio IDE displays in iframe and works in popout
Open Code/1_credit_permissiveness.R
Confirm outputs are created in Outputs without error

Compose Jupyter Notebook Tale

Create tale from Git repo https://github.com/whole-tale/jupyter-example/
Select environment Jupyter Notebook
Run tale and confirm Jupyter Classic IDE displays in iframe and works in popout
Add usco2005.xlsx from Humans and Hydrology dataset
Run notebook, confirm it produces most glorious map of Texas

Compose JupyterLab Tale

Create tale from Git repo https://github.com/whole-tale/jupyter-example/
Select JupyterLab environment
Run tale and confirm JupyterLab IDE displays in iframe and works in popout
Confirm JupyterLab IDE displays
Add usco2005.xlsx from Humans and Hydrology dataset
Run notebook, confirm it produces most glorious map of Texas

Compose JupyterLab with Spark

Create tale from Git repo https://github.com/whole-tale/spark-example/
Select Jupyter with Spark environment
Run tale and confirm JupyterLab IDE displays in iframe and works in popout
Run notebook and confirm version is 2.4.3

Compose MATLAB Desktop Tale

Create tale from Git repo https://github.com/whole-tale/matlab-example/
Select MATLAB Desktop environment
Run tale and confirm MATLAB Web Desktop IDE displays in iframe and works in popout
Open and run multiplicative_arima_example_script.m, confirm outputs
Open and run multiplicative_arima_example_livescript.m, confirm outputs

Compose MATLAB Jupyter Kernel Tale

Create tale from Git repo https://github.com/whole-tale/matlab-example/
Select MATLAB Jupyter Kernel environment
Run tale and confirm JupyterLab IDE displays in iframe and works in popout
Open and jun multiplicative_arima_example.ipynb, confirm outputs match multiplicative_arima_example.html

Compose MATLAB Linux Desktop Tale

(Only test if present)
Create tale from Git repo https://github.com/whole-tale/matlab-example/
Run tale and confirm Xpra-based MATLAB IDE displays in iframe and works in popout
Open and run multiplicative_arima_example_script.m, confirm outputs

Compose STATA Desktop Tale

Create tale from Git repo https://github.com/whole-tale/stata-example/
Run tale and confirm Xpra-based STATA IDE displays in iframe and works in popout
Run example.do, confirm outputs

Compose STATA Jupyter Kernel Tale

Create tale from Git repo https://github.com/whole-tale/stata-example/
Run tale and confirm JupyterLab IDE displays in iframe and works in popout
Run example.ipynb, confirm outputs match example.html

Compose OpenRefine Tale

Create a new tale based on the OpenRefine environment
Run tale and confirm OpenRefine IDE displays in iframe and works in popout

Too many instances

Run 2 tales
Attempt to run another Tale
Confirm that you receive an error that you have the max limit

Analyze in Whole Tale

These test cases cover potential situations that can occur when importing datasets from Dataverse.

Import dataset from Dataverse

Remove all running Tale instances
Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataverse?datasetPid=doi%3A10.7910%2FDVN%2F3MJ7IR&siteUrl=https%3A%2F%2Fdataverse.harvard.edu
Confirm that the Tale title reads as Replication Data for: "Agricultural Fires and Health at Birth"
Confirm that the only item in the Input data section matches the uri with Data Source appended
Confirm that no environment is selected
Confirm that the Create New Tale button is disabled
Select an environment
Confirm that READ/WRITE is selected
Click Create New Tale
Confirm registration notification progress and success
Confirm that you are redirected to the Metadata page and "Run Tale" is enabled
Confirm that the Tale name matches the Tale Name in the Create Tale Modal
Confirm that the data exists under Tale Workspace
Confirm that the Tale's category is science

Import dataset from DataONE: READ-ONLY

Remove all running Tale instances
Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataone?uri=https%3A%2F%2Fsearch.dataone.org%2Fview%2Fdoi%3A10.18739%2FA2VQ2S94D&title=Fire%20influences%20on%20forest%20recovery%20and%20associated%20climate%20feedbacks%20in%20Siberian%20Larch%20Forests%2C%20Russia&environment=RStudio
Confirm that the Tale title reads as Fire influences on forest recovery and associated climate feedbacks in Siberian Larch Forests, Russia
Confirm that the only item in the Selected data section matches the uri with Data Source appended
Confirm that Rstudio is selected in the Environments widget
Confirm that the Create New Tale button is enabled
Confirm READ ONLY is selected
Click Create New Tale
Confirm that you are redirected to the Metadata tab
Confirm that the Tale name matches the Tale Name in the Create Tale Modal
Confirm that the data exists in the Tale under External Data
Confirm that the Tale's category is science
Confirm that the Datasets Used and Related Identifiers are updated to reflect the dataset

Import from DataONE: READ-WRITE

Navigate to https://girder.stage.wholetale.org/api/v1/integration/dataone?uri=https%3A%2F%2Fsearch.dataone.org%2Fview%2Fdoi%3A10.18739%2FA2VQ2S94D&title=Fire%20influences%20on%20forest%20recovery%20and%20associated%20climate%20feedbacks%20in%20Siberian%20Larch%20Forests%2C%20Russia&environment=RStudio
Confirm that the Tale title matches the dataset
Confirm that the only item in the Selected data section matches the uri with Data Source appended
Confirm that the Create New Tale button is disabled
Select READ/WRITE
Click Create New Tale
Confirm that the notification bar appears & properly updates
Confirm that you are redirected to the run page
Confirm that the Tale name matches the Tale Name in the Create Tale Modal
Confirm that the data exists in the Tale Workspace

Import from DataONE: alternate sites
1. "proveit": https://dashboard.stage.wholetale.org/mine?api=https%3A%2F%2Fcn-stage-2.test.dataone.org%2Fcn%2Fv2&name=Dataone%20Dataset&uri=https://dev.nceas.ucsb.edu/view/doi:10.5072/FK2K075M25
2. "Alpha Helix 2001...": https://dashboard.stage.wholetale.org/mine?api=http%3A%2F%2Fcn-stage.test.dataone.org%2Fcn%2Fv2&name=Dataone%20Dataset&uri=https://test.arcticdata.io/view/doi:10.5065/D6N014NC
3. Remove all running Tale instances
4. Navigate to each of the two links above
5. Select an environment
6. Click Create New Tale
7. Confirm that the correct data exists in the Tale under External Data
Import from Zipfile
1. Export a tale you own or copy of LIGO tale
2. Import via API

export GIRDER_TOKEN=<your token>
curl -X POST -H "Content-Type: application/zip" --data-binary "@<path-to-zip>.zip" --header "Girder-Token: ${GIRDER_TOKEN}" 'https://girder.stage.wholetale.org/api/v1/tale/import'

Confirm tale imports, builds and run
Try with invalid token: (You must be logged in)
Try with invalid zipfile (Bad bag)

Tale metadata tests

The purpose of these tests are to confirm that the metadata files (manifest.json, environment.json, LICENSE) we generate are correct.

manifest.json

Launch a Tale that you own
Add a dataset to the Tale
Add at least one author
Add at least one file to the workspace
Create a version
Export the Tale (as BagIt)
Open the manifest.json file under metadata/
Confirm wt context present
Confirm @id references the correct Tale ID
Confirm that the author is listed under schema:author
Confirm that you are listed under createdBy
Confirm that the file(s) in the workspace are properly listed in aggregates
Confirm that the external dataset is referenced in aggregates
Confirm that the dataset is also listed under wt:usesDataset
Confirm dct:hasVersion is present and matches your created version
Optionally confirm rdflib can parse:

from rdflib import Graph

import urllib.parse
urllib.parse.uses_relative.append('arcp')
urllib.parse.uses_netloc.append('arcp')

g = Graph().parse(source="manifest.json", format="json-ld")
print(g.serialize(format='turtle', indent=2).decode())

environment.json

Open the archive from the manifest.json test
Navigate to metadata/environment.json
Open the girder web API
Locate and GET the Image that the Tale used
Confirm that the config section from girder matches the config section in environment.json

LICENSE

Open the archive from the manifest.json test
Open the LICENSE file
Navigate to Run > metadata
Confirm that the opened LICENSE file matches the selected license on the metadata page
Change the Tale's license
Export the Tale
Confirm that the LICENSE changed appropriately

Tale Sharing Tests

These tests are centered around the feature for sharing Tales with other Whole Tale
users. To run these tests, register at least one other account, referred to as User B.

Case 1: Sharing with 'Can View' permissions
1. Create a new Tale
2. Add data to the workspace and add at least one dataset
3. Navigate to the 'Share' tab
4. Share the Tale with User B
5. Confirm that the default permission is 'Can View'
6. Switch the account to User B
7. Navigate to the 'Shared with Me' tab on the Browse page
8. Confirm that you see the Tale from step 1
9. Confirm that you can't edit the Workspace, External Data, or metadata
10. Unshare tale with User B
11. Note modal displays that tale has been unshared.
Case 2: Sharing with 'Can Edit' permissions
1. Create a new Tale
2. Add data to the workspace and add at least one dataset
3. Navigate to the 'Share' tab
4. Share the Tale with User B
5. Set the permissions to 'Can Edit'
6. Switch the account to User B
7. Navigate to the 'Shared with Me' tab on the Browse page
8. Confirm that you see the Tale from step 1
9. Remove one of the files from the Workspace
10. Add a dataset to External Data
11. Change a field in the metadata (title, description, etc)
12. Switch to User A
13. Confirm that the changes made to the Tale persisted
Case 3: Edit sync modal
1. Login as User A and User B in separate browser sessions
2. As User B, select "Shared with me"
3. As User A, share a tale with User B; as User B confirm the tale appears
4. As User B, select "View" on the shared tale
5. As User A and B edit the tale metadata
6. As User A change title, author, category, description; save changes
7. As User B confirm Tale edited modal, select Yes, confirm updates to metadata

Publishing tests

Case 1: No Accounts Configured
1. Disconnect all of your external accounts
2. Open a Tale that you own
3. Click the ellipsis dropdown on the Run page
4. Click Publish tale...
5. Confirm that your are notified to link accounts on the settings page
6. Click Go to Settings
7. Confirm you are re-directed to settings
Case 2: Publishing to DataONE
1. Connect to DataONE (dev server)
2. Open a Tale that you own
3. Add some data to it
4. Click Publish tale...
5. Confirm that the publishing modal opens
6. Confirm that the third party that you are connected to appears in the dropdown
7. Select the third party
8. Click Publish
9. Once complete, navigate to the metadata page
10. Confirm that the published location is present
11. Visit the published location
12. Confirm that you see the following files plus yours
  1. metadata.xml
  2. LICENSE
  3. README.md
  4. manifest.json
  5. environment.json
13. After publishing to DataONE, use girder to query the Tale
14. Ensure that the 'pid' field in the 'publishInfo' object has an identifier in a doi format
Case 3: Re-publish Tale to DataONE

Launch a Tale that has been published before
Take note of the Published Location in Run > metadata
Re-publish the Tale to the third party
Confirm that the Published Location has changed to the correct package landing page

Zenodo integration tests

Register Zenodo data

On a new or existing tale, select "Files" > "External Data"
Select "+" > "Web (DOI or URL)"
Search for Concept DOI doi:10.5281/zenodo.16384
Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_1
Search for URL https://zenodo.org/record/1172960#.XidIvlNKh24
Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_2
Search for DOI doi:10.5281/zenodo.1172960
Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_2
Search for Concept DOI doi:10.5281/zenodo.608984
Confirm "Belmont Forum..." dataset is found on Zenodo with suffix ver_2
Select Register
Select "+" > "WT Data Catalog"
Add "Belmot Forum..." dataset to Tale
Run Tale and confirm contents on file system match Zenodo

The register tests the following cases.

Datasets with multiple files
Datasets with multiple versions
Concept DOI
Zenodo dataset via AiWT

Open https://dashboard.stage.wholetale.org/mine?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.820575&name=Automotive%20Sensor%20Data
Confirm Source Data URL and Title match above URL and "READ ONLY" is selected
Select JupyterLab environment
Select 'Create New Tale'
Confirm dataset is mounted read-only, citation and related ids are as expected,
Run IDE and use terminal to ls ../data/Auto* to confirm zipfile is not extracted

The register tests the following cases.

Dataset with single file (zipfile)
Zenodo Binder via AiWT

Open https://dashboard.stage.wholetale.org/mine?uri=https%3A%2F%2Fdoi.org%2F10.5281%2Fzenodo.3242073&environment=JupyterLab&name=Simple%20requirements.txt%20based%20example%20for%20repo2docker&asTale=true
COnfirm Source Data and Title match URL and "READ WRITE" is selected
Select 'Create Tale'
Confirm image builds and notebook can run

Zenodo Tale via AiWT

Open https://girder.stage.wholetale.org/api/v1/integration/zenodo?doi=10.5281%2Fzenodo.490449&resource_server=sandbox.zenodo.org
Confirm Water Tale is created.
Confirm citation on Metadata view
Confirm DOI matches above
Use the Swagger UI to confirm the publishInfo field on the tale object contains pid, repository, repository_id, uri
Run the tale and and notebook, confirm it works as expected (produces glorious map of Texas)

Publish/import to/from Zenodo Sandbox

Follow instructions above to configure your API key for Zenodo sandbox
Create your own Tale and publish to sandbox.zenodo.org
Confirm publishInfo contains Sandbox entry
Delete tale
Import tale from Zenodo
Confirm publishInfo
Modify the tale
Re-publish to sandbox.zenodo.org
Delete tale
Import tale from Zenodo
Confirm publishInfo
Download and extract the exported Tale
Execute the run-local.sh command to run the Tale locally

OpenICPSR integration tests

OpenICPSR via AinWT

Go to Settings, configure ICPSR provider. Select "www.openicpsr.org". Provide you password for icpsr login. (Assumption: your globus/girder account uses the same email as icpsr entity.)
Go to: https://dashboard.stage.wholetale.org/mine?name=AEAREP-3198-Stata&environment=STATA%2016%20%28Desktop%29&uri=https%3A%2F%2Fwww.openicpsr.org%2Fopenicpsr%2Fproject%2F132081%2Fversion%2FV1%2Fview&asTale=true
Confirm import

Register OpenICPSR dataset?

This isn't an officially supported use case, but register https://www.openicpsr.org/openicpsr/project/132081/version/V1/view and confirm it works.

Git Integration Tests

Compose Tale from Git Repository
Covered by other tests

Select the Create New Tale button on the 'Browse' page
Select the Create Tale from Git Repository option
Paste a link to a Git repository in the correct field
Add a name and pick a compute environment
Create the Tale
Confirm notification progress and success
Confirm that the git repository was cloned to the workspace/ root
Run the tale
Confirm interaction via git tools (may require environment customization)

Connect to Git

Create a new empty tale
Select "Connect to Git Repository" menu item from Tale menu
Confirm notification progress and success
Once complete, navigate to the Tale workspace and confirm that the repository has been cloned

Other tests

Citations

Add files from multiple external datasets (e.g.: doi:10.5281/zenodo.3381658, doi:10.5065/D6862DM8, doi:10.5063/F10K26VR)
Confirm Datasets Used citations and related identifiers are updated

Event Notification Tests

Tale sharing/unsharing

Login as user1 and user2 in separate browser sessions
As user2, select "Shared with me"
As user1, share a tale with user2; as user2 confirm the tale appears
As user2, select "View" on the shared tale
As user1 and 2 edit the tale metadata
As user1 save changes; as user2 confirm Tale edited modal
As user1, unshare the tale; as user2 confirm Tale unshared modal
As user1, re-share with user2
As user2, select "Shared with Me"
As user1, delete the shared tale; as user2 confirm that the tale disappears

Import tale

Import a tale from Github, Zenodo, and as Binder
Confirm metadata updates accordingly after import completes

Vocabulary Tests

Vocabularies

Go to https://vocabularies.stage.wholetale.org/wt/1.0/index.html
Confirm that it matches https://whole-tale.github.io/serialization-format/wt/1.0/

Forward Authentication Test

Forward authentication

Start a tale
Confirm that the tale is accessible via iframe and popout
Right click popout icon and copy URL (this is necessary to get JupyterLab token)
Open an incognito window and paste the copied URL
Confirm that you are prompted for authentication and can access the tale once authenciated
Login as a user who does not have permission to access the tale
Confirm that they receive an access denied errror

Changing the Environment

Changing the environment

Create a new JupyterLab tale
Run the Tale, confirm JupyterLab IDE displays
Edit metadata, change environment to "RStudio"
Confirm icon changes in title bar and Environment section
Stop/run tale or rebuild/restart and confirm RStudio IDE displays

Idle Instances

Idle instance culling

Create and start a MATLAB tale and a JupyterLab tale
Confirm both instances start and IDEs are accessible
Close all IDE tabs/windows
Wait for 2 hours and 24 hours respectively
Confirm both instances are stopped automatically

Upload folder

Upload folder to home

Create local folder containing files, folders, etc.
Upload folder to home. Confirm contents match local folder.
Download folder. Confirm contents match.
Repeat in workspace

Advanced settings

Advanced settings

Create a new tale
Edit Metadata > Advanced, enter:

{
    "environment": [
        "MY_ENV=value"
    ],
    "memLimit":"12gb"
}
1. Start tale
1. `cat /sys/fs/cgroup/memory.max`, confirm 12gb
1. `echo $MY_ENV`, confirm value

Create Tale from DOI

Create Tale from DOI

#
Create Tale > Create Tale from DOI
Enter DOI doi:10.7910/DVN/TJCLKP: Title should populate with the DVN dataset title
Enter URL https://sandbox.zenodo.org/record/1059441: Modal should convert to import tale
Enter a non-existent DOI, confirm that Failed to find DOI/URL message displays
Confirm that radio buttons work as expected
Logout, try each of the following via AinWT

Public image registry

Public image registry

Create an empty tale
Export tale without running it
Inspect and run run-local.sh. Image should be built locally using r2d.
On WT, run your tale (building the image)
Export tale
Inspect and run run-local.sh. Image should be pulled from images.stage.wholetale.org
Publish the tale to sandbox.zenodo.org
Delete the tale from WT
Import the tale
Export the tale from WT, confirm correct image in run-local.sh
Download the tale from Zenodo, confirm image in run-local.sh matches

The text was updated successfully, but these errors were encountered:

craig-willis · 2022-09-19T16:26:16Z

Issues:

Tale history panel default size is too small -- Default height of Tale Version Panel is too small ngx-dashboard#290
Home directory doesn't refresh on create folder? -- Folder not displayed after create ngx-dashboard#299
Zenodo integration link fails if login via ORCID due to state parameter limit. Any AinWT fails with ORCID if URL is too long (multiple test cases). -- 500 error when logging in with ORCID girder_wholetale#534
Version menus are inconsistent -- Consolidate version operations ngx-dashboard#294
Recorded run doesn't update tale imageInfo if building image, which results in run local pulling wrong image -- Recorded run image build issues gwvolman#181
OpenICPSR import succeeds, but files contain errors "Unrecoverable error in the server." -- OpenICSPR import results in multiple files girder_wholetale#555
DERIVA and Globus fetch fails on run local -- BDBag fetch support for DERIVA zips whole-tale#114
~~Jupyter Example failing to build because "ImportError: cannot import name 'Markup' from 'jinja2'" during postBuild~~ (fixed in example repo by p)
DataONE integration doesn't handle asTale case?
Settings > DataONE > Connect Account getting bad cert on stage-2 CN. Needed to change to cn-stage.
DataOne: Create version > publish; recorded run > publish -- fails because version already published. However, changing metadata and creating a version and publishing ... there's no run in the zipfile?
Zenodo Binder via AiWT fails with "Copying files to workspaceCreateFailed: CreateFailed('unable to create filesystem, File is not a zip")
Pyspark example failing: Import pyspark fails during Spark/Jupyter test (TypeError: an integer is required (got type bytes))

Can't repeat:

Interact frame doesn't always display the IDE when I'm not looking? No spinner, no IDE, but it opens fine on refresh or popup.
Also more oddities with instance state and notifications. Stepped away while spark/matlab images built and came back to them in progress but logs unviewable. After refresh both appeared with "stop tale" on catalog but not in "Currently running". But both IDEs were accessible.
Signing out while viewing a private tale still shows me the tale metadata page

Observations:

Remove menu option displays on versions/runs for user without owner permissions, but error displays when I try to remove it.
AinWT with "browse" target fail silently. Can we redirect to "mine"?
"Tale Dashboard" could just be "Dashboard" and should match "wholetale.org" header
Having a message that you need to # would be nicer than the tooltip over the run button
DERIVA not listed on register data, but that's probably OK
Do we really need to test travis.yml?
I still wish ORCID field was autopopulated or at least remembered by browser
"Upload folder (TODO)" (test case needs writing)
Should we just add bdbag validation to run-local?
It would be nice to be able to see versions/runs in Jupyter without using terminal
Need test case for imageInfo?
What happens when DV token expires or is revoked?
"Create JupyterLab Tale" is redundant
Do we need to add RStudio / MATLAB / STATA versions with this release?
Too bad we don't have a way to add datasets to a Git repo... (Jupyter Example)
There isn't an easy way to pull a Git repo if the one I imported fails to build. I must delete and recreate or remove r2d config (another case for "Safemode"?)
Remove openrefine test?
MATLAB Jupyter test warns "Warning: The system time zone setting, '/UTC', is unrecognized and 'UTC' will be used instead. Accessing the current time, e.g. datetime('now'), or setting the time zone property of a datetime array, e.g. datetime(Y,M,D,'TimeZone','local'), might return incorrect results."
Remove DataONE alternate sites test case?
"Run Tale" should really be "Copy Tale" if you can't run it, right?
Automate tale metadata test... or do we really need it?
It would be nice to have access to the container instance logs while my session is running. Also should keep a copy of the build log as part of the exported/published tale.

This was referenced Sep 26, 2022

Recorded run image build issues whole-tale/gwvolman#181

Open

Can't republish tale version after recorded run whole-tale/gwvolman#182

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v1.2rc1 testing (Chrome/MacOS) #171

v1.2rc1 testing (Chrome/MacOS) #171

craig-willis commented Sep 19, 2022 •

edited

Loading

craig-willis commented Sep 19, 2022 •

edited

Loading

v1.2rc1 testing (Chrome/MacOS) #171

v1.2rc1 testing (Chrome/MacOS) #171

Comments

craig-willis commented Sep 19, 2022 • edited Loading

Test Plan

End-to-End Test

Unauthenticated access

Signing in

Navigation

Tale Dashboard

Managing Data

Run Tale

Versions

Recorded Runs

Settings

Tale Creation

Analyze in Whole Tale

Tale metadata tests

Tale Sharing Tests

Publishing tests

Zenodo integration tests

OpenICPSR integration tests

Git Integration Tests

Other tests

Event Notification Tests

Vocabulary Tests

Forward Authentication Test

Changing the Environment

Idle Instances

Upload folder

Advanced settings

Create Tale from DOI

Public image registry

craig-willis commented Sep 19, 2022 • edited Loading

craig-willis commented Sep 19, 2022 •

edited

Loading

craig-willis commented Sep 19, 2022 •

edited

Loading