Adding ocean-modelling-litter-philab notebook #22

raquelcarmo · 2021-11-26T14:41:09Z

Before review

Work in progress
Ready for review
Need help!

Description

This is the first draft of the Marine Litter Detection notebook.

Motivation and Context

How has this been tested?

Locally.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
New notebook and/or content
Documentation notebook/content update
Other (please describe):
Feature change (upgrade version)

Checklist:

My code follows the code style of this project: Python
I have read the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added tests to cover my changes
I have tested the changes and verified that they work and don't break anything (as well as I can manage)

Additional information

Added litter philab notebook

review-notebook-app · 2021-11-26T14:41:13Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

netlify · 2021-12-05T17:08:39Z

✔️ Deploy Preview for the-environmental-ds-book ready!

🔨 Explore the source changes: be72357

🔍 Inspect the deploy log: https://app.netlify.com/sites/the-environmental-ds-book/deploys/61f41c6b1777e20008c5e0d5

😎 Browse the preview: https://deploy-preview-22--the-environmental-ds-book.netlify.app/ocean/modelling/ocean-modelling-litter-philab

review-notebook-app · 2021-12-05T21:00:31Z

View / edit / reply to this conversation on ReviewNB

acocac commented on 2021-12-05T21:00:31Z
----------------------------------------------------------------

I updated the context header section with additional metadata according to the latest template. The main comments are:

Modelling codebase: where possible provide the complete list of contributors. You can differentiate contributors in two groups: main authors (those who have considerable contributed to the codebase), and contributors (those who have found/fixed bugs or contribute with a particular enhancement from the original codebase).
Modelling publications: this section lists related publications according to the references.bib file. Please confirm if the existing publications are ISPRS Conference Paper and OCEANs.
Modelling funding: only complete this section if the research has been part of any particular funding.
:::{note} The notebook contributors acknowledge ... ::: this section could be complete in the final draft. I would suggest to give credits to ESA Sentinel-2 mission and/or other data providers relevant in your work.

raquelcarmo commented on 2021-12-14T15:24:30Z
----------------------------------------------------------------

Thanks for the feedback! Will add another main author to our list. The publications are indeed the ISPRS and OCEANS.

acocac commented on 2021-12-17T17:33:31Z
----------------------------------------------------------------

Cool! I confirm I saw the update of the codebase authorship. Thanks for confirming the existing publications.

acocac commented on 2022-01-07T19:02:10Z
----------------------------------------------------------------

@raquelcarmo I've pushed some substancial changes to the notebook. Following my initial suggestions, I strongly recommend to focus the notebook on modelling rather than data preprocessing. If this is ok for you, it would be great to have a separate notebook showcasing the removed section of data preprocessing. Please find further details of the changes in the PR. I'm highlight below some ideas to improve the content of the proposed notebook.

review-notebook-app · 2021-12-05T21:00:32Z

View / edit / reply to this conversation on ReviewNB

acocac commented on 2021-12-05T21:00:31Z
----------------------------------------------------------------

Have you considered to mirror the pretrained models in a more findable and persistent location? For instance, I suggest to upload the pretrained models to Zenodo. You can add metadata of each them. Another plus of Zenodo, is that you can link your data/software repositories to communities. For instance, I recently start curating some datasets of the Environmental DS book within Environmental DS community in zenodo.

review-notebook-app · 2021-12-05T21:00:33Z

View / edit / reply to this conversation on ReviewNB

acocac commented on 2021-12-05T21:00:32Z
----------------------------------------------------------------

I suggest to add further description of the differences between the models. You can also highlight the most relevant one. For instance, you can mention there are X number of pretrained models available (describe in this link), but for sake of simplicity you are loading the weights of the 'unet_seed0'. The loaded model refers to a U-Net pre-trained architecture with for 50 epochs, with batch size of 160, learning rate of 0.001, seed 0.

review-notebook-app · 2021-12-05T21:00:33Z

View / edit / reply to this conversation on ReviewNB

acocac commented on 2021-12-05T21:00:33Z
----------------------------------------------------------------

My suggestion is to focus the notebook with analysis-ready data (stacks) generated for certain locations (at least 5 areas) of your research. You can upload the stacks in Zenodo and indicate in the same repository how you download the original dataset .e.g SentinelHub and preprocess it (i.e. mentioning the python packages). It's ok to add a disclaimer or note to readers that they can also test in their own locations with similar stacks generated from their preferred platform (sentinelhub, gee, planetary computer).

review-notebook-app · 2021-12-05T21:00:34Z

View / edit / reply to this conversation on ReviewNB

acocac commented on 2021-12-05T21:00:34Z
----------------------------------------------------------------

Please comment if the model also works for other processing levels of S2 e.g. TOA or fusion-based products e.g. Harmonized Landsat Sentinel-2

jmifdal commented on 2021-12-17T11:17:28Z
----------------------------------------------------------------

Good suggestion, we'll comment this part

review-notebook-app · 2021-12-05T21:00:35Z

View / edit / reply to this conversation on ReviewNB

acocac commented on 2021-12-05T21:00:34Z
----------------------------------------------------------------

I would add further details of the relevance of NDVI and FDI for identifying floating objects. Are they relevant to the model? or only for visual inspection?

raquelcarmo commented on 2021-12-14T15:26:22Z
----------------------------------------------------------------

Thanks for the feedback, will add this!

acocac commented on 2021-12-17T17:31:06Z
----------------------------------------------------------------

Raquel, you're welcome. It's fair to provide some context thinking in the audience of the Environmental Data Science book.

review-notebook-app · 2021-12-05T21:00:35Z

View / edit / reply to this conversation on ReviewNB

acocac commented on 2021-12-05T21:00:35Z
----------------------------------------------------------------

Nice viz! We can make it more interactive using hvplot. I could provide an example of code to generate the interactive plot when I get access to the analysis ready data. Here's an example of a RGB image interactively plotted using hvplot.rgb

review-notebook-app · 2021-12-05T21:00:36Z

View / edit / reply to this conversation on ReviewNB

acocac commented on 2021-12-05T21:00:36Z
----------------------------------------------------------------

Great example. Would it be ok to add the link to the original URL of the scene? or maybe can you ask to the Plastic Litter Project (PLP) if you can mirror the dataset in Zenodo?

jmifdal commented on 2021-12-17T11:15:20Z
----------------------------------------------------------------

Thanks Alejandro for the comment, we'll add the link to the original scene.

acocac commented on 2021-12-17T17:30:00Z
----------------------------------------------------------------

Thanks Jamila. Looking forward to reviewing the updated version.

review-notebook-app · 2021-12-05T21:00:37Z

View / edit / reply to this conversation on ReviewNB

acocac commented on 2021-12-05T21:00:36Z
----------------------------------------------------------------

Really nice work and great contribution to the ocean domain! Please add a Summary section to highlight the main conclusions of your research and open-source packages explored in the notebook.

acocac · 2021-12-05T21:38:35Z

@raquelcarmo @jmifdal thanks for the contribution!! I went through the first draft and it looks great! I added some comments via ReviewNB. Please be free to reply them and implement the suggested changes if you agree.

I also pushed the following changes:

Update the context header according to the latest modelling template.
Add existing references of your work to the references.bib file (see L183 in the changed file). These publications will render in the Modelling publications section of the notebook.

Finally, some additional thoughts:

While it's great to show how to retrieve satellite imagery from the SentinelHub python package, I think the main goal of the notebook should be to highlight how your models work to detect floating objects from S2 imagery. My main suggestion is to create a Zenodo repository to mirror stacked images (so called analysis-ready data) of a given number of locations of your preference. You can also upload to the same Zenodo repository files of the pretrained models.
For example, this Zenodo repository was successfully incorporated within a notebook showcasing a Deep Learning model to detect tree crowns from drone imagery.
If you create the zenodo repository, you can link it to the Environmental DS community.

I hope the above suggestions, changes and thoughts are helpful for improving the content and structure of the notebook.

raquelcarmo · 2021-12-14T15:24:31Z

Thanks for the feedback! Will add another main author to our list. The publications are indeed the ISPRS and OCEANS.

View entire conversation on ReviewNB

jmifdal · 2022-01-14T12:23:04Z

Thank you Alejandro. I had some issues with the installation of "intake", I had to install the "pycurl" package and run these two lines: "!conda install -c conda-forge intake-xarray" and "conda install -c conda-forge pydap rasterio", I hope that this is taken into account in the EnvDS book.

View entire conversation on ReviewNB

jmifdal · 2022-01-14T12:25:58Z

Thanks you Alejandro, it's a good idea. We exported the predictions of Rio de Janeiro into GeoTIFF format and we also exported the use-case predictions into tiff formats.

View entire conversation on ReviewNB

jmifdal · 2022-01-14T12:26:58Z

We changed the sentence above and we made sure to mention that the data is fetched from Zenodo.

View entire conversation on ReviewNB

jmifdal · 2022-01-14T12:27:30Z

Done :-)

View entire conversation on ReviewNB

jmifdal · 2022-01-14T12:36:20Z

Thank you very much @acocac for the relevant review. Normally we took into account all your remarks. We agree with the fact that the notebook should be split into two parts and we would happily contribute with another notebook highlighting the preprocessing steps.

We will push the new version of the notebook soon, let us know if you have any further remarks.

Thanks

Best,

Raquel and Jamila

acocac · 2022-01-14T16:10:54Z

@jmifdal @raquelcarmo glad to hear the feedback is useful (: I don't have more remarks. After the push, the notebook status will change to ready for review. I was suggested to ask Claire Waluda her availability for a domain-expert review of the notebook and research in general. Let me know if you're keen to have her feedback. Otherwise, it would be ok to publish the notebook.

acocac · 2022-01-28T15:22:30Z

@raquelcarmo @jmifdal Please have a look at the preview of the notebook.

I just added the citation and badge of the RoHub research object created for the notebook.

Note we can also specify AOIs associated to the research object. If it doesn't take long for you, please send me GeoJSON files of all prediction areas. Then I can add them to the research object.

netlify · 2022-01-28T19:25:22Z

A reviewer left a comment:

I'm sharing the video below showing the RoHub badge. Note the link to the notebook will be update with the link in the GitHub repo of the EnvDS book. The last section also includes how to cite the notebook.

ps. I've really enjoyed the experience to help reviewing your great work!! The new features such as the citation and RoHub are useful for existing notebooks and future publications. Thanks for contributing!

Browser metadata

Path:      /ocean/modelling/ocean-modelling-litter-philab#citing-this-notebook
Browser:   Chrome 96.0.4664.110 on Mac OS 10.15.7
Viewport:  1440 x 821 @2x
Language:  en-GB
Cookies:   Enabled

Open in BrowserStack

Open Deploy Preview · Mark as Resolved

raquelcarmo added 4 commits November 26, 2021 11:45

Created modelling folder

931eba6

Added litter philab notebook

7332f73

Merge pull request #1 from raquelcarmo/raquelcarmo-notebook

930a3b3

Added litter philab notebook

Delete dummy.txt

ae3adbe

acocac self-requested a review November 26, 2021 16:43

acocac assigned raquelcarmo Nov 26, 2021

acocac added notebook Add/update notebook modelling Modelling Notebooks high priority labels Nov 26, 2021

acocac added this to the 0.0.2 milestone Dec 2, 2021

remove section in title

bd67422

acocac added 3 commits December 5, 2021 17:10

add section badges

b1ead66

update the context header with the latest template

275a98e

add publications

7632a81

acocac removed this from the 0.0.2 milestone Dec 8, 2021

Update ocean-modelling-litter-philab.ipynb

7d3476f

acocac added 3 commits January 20, 2022 09:57

merge cells for hide-input

32def64

fix references

4083708

add booktitle in references

d8a6327

acocac assigned jmifdal Jan 20, 2022

raquelcarmo and others added 9 commits January 27, 2022 11:23

Updated modelling funding message

7b3c515

remove acknowledgment note

fad5e21

fix typos

45dd834

change binderhub

6ec3306

change title size and export hvplot

331f26f

remove tiff format

a4f1664

add rohub citation and badge

f8df546

edit RoHub badge

58314fe

hide view metadata

7e702d6

acocac added 4 commits January 28, 2022 15:52

update citation

64be462

update url RoHub RO

1b3e99c

update RO type in RoHub badge

eb996c5

update RO type in RoHub badge

be72357

acocac merged commit a959499 into alan-turing-institute:master Jan 30, 2022

acocac added the ready label Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding ocean-modelling-litter-philab notebook #22

Adding ocean-modelling-litter-philab notebook #22

raquelcarmo commented Nov 26, 2021 •

edited by acocac

Loading

review-notebook-app bot commented Nov 26, 2021 •

edited by acocac

Loading

netlify bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

acocac commented Dec 5, 2021 •

edited

Loading

raquelcarmo commented Dec 14, 2021

jmifdal commented Jan 14, 2022

jmifdal commented Jan 14, 2022

jmifdal commented Jan 14, 2022

jmifdal commented Jan 14, 2022

jmifdal commented Jan 14, 2022

acocac commented Jan 14, 2022 •

edited

Loading

acocac commented Jan 28, 2022

netlify bot commented Jan 28, 2022

Adding ocean-modelling-litter-philab notebook #22

Adding ocean-modelling-litter-philab notebook #22

Conversation

raquelcarmo commented Nov 26, 2021 • edited by acocac Loading

Before review

Description

Motivation and Context

How has this been tested?

Types of changes

Checklist:

Additional information

review-notebook-app bot commented Nov 26, 2021 • edited by acocac Loading

netlify bot commented Dec 5, 2021 • edited Loading

review-notebook-app bot commented Dec 5, 2021 • edited Loading

review-notebook-app bot commented Dec 5, 2021

review-notebook-app bot commented Dec 5, 2021 • edited Loading

review-notebook-app bot commented Dec 5, 2021 • edited Loading

review-notebook-app bot commented Dec 5, 2021 • edited Loading

review-notebook-app bot commented Dec 5, 2021 • edited Loading

review-notebook-app bot commented Dec 5, 2021 • edited Loading

review-notebook-app bot commented Dec 5, 2021 • edited Loading

review-notebook-app bot commented Dec 5, 2021 • edited Loading

acocac commented Dec 5, 2021 • edited Loading

raquelcarmo commented Dec 14, 2021

jmifdal commented Jan 14, 2022

jmifdal commented Jan 14, 2022

jmifdal commented Jan 14, 2022

jmifdal commented Jan 14, 2022

jmifdal commented Jan 14, 2022

acocac commented Jan 14, 2022 • edited Loading

acocac commented Jan 28, 2022

netlify bot commented Jan 28, 2022

raquelcarmo commented Nov 26, 2021 •

edited by acocac

Loading

review-notebook-app bot commented Nov 26, 2021 •

edited by acocac

Loading

netlify bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

review-notebook-app bot commented Dec 5, 2021 •

edited

Loading

acocac commented Dec 5, 2021 •

edited

Loading

acocac commented Jan 14, 2022 •

edited

Loading