[chore] Webscraper Fixes for NYISO and NYSERDA

[itsliterallymonique]

This week I’ll have you make some changes and fixes to the NYISO and NYSERDA. Here is what you’ll do:

1. NYISO
   a. You will also need to scrape the utility service provider column from the dataset. This will be in the “Utility” column. The name in our database should be utility_service_provider. Make sure to also reflect this in our Project schema in our repo.
   b. We will no longer be scraping data from the “Interconnection Queue” as that is inaccurate. Instead, we should only use “Cluster Projects” and “In Service”
2. NYSERDA
   a. If a project’s status in NYSERDA is listed as “Cancelled”, you will still need to pull it from the database and re-label it as “Proposed”. This is because NYSERDA mass “cancelled” a bunch of projects that are still in progress and are going through the process of re-approving all of them.
3. For both NYSERDA and NYISO, make sure when we update the dataset we don’t re-add projects that have been deleted from the NPO! Hint: check the last_updated column.
4. KDMs
   a. If “operations begun” is TRUE, everything else should be set to true! This is the last KDM checkpoint
   b. The operations begun date is also the operational COD date. So make sure the COD is set to this operational date as well.
5. Check we are omitting projects that are not in New York State!!