Spam-report for juice-shop pull requests and issues

[hxrshxz]

• Add metric for spam vs total PR count #28

Description

Spam issues and Pull requests report graph for the Juice-shop app

Screencast.from.2025-02-22.03-06-25.mp4

[bkimminich]

Retrieving the last 30 days every time is not the way to go. Take a look at the other GitHub stats, we pull the data of the previous day and append it to a JSON, then commit to the repo. This means we have a growing permanent history. For the initial JSON you'd need to run the script for each day starting with the first day that ever had a spam PR/issue, then concatenate all subsequent days except the day that would be appended by the next CI run.

Also, the visualization is missing, and without the PR is too incomplete to merge.

Lastly, please remove your GitHub token. I think it works without - see the other GitHub stats again: https://github.com/juice-shop/juicy-statistics/blob/master/extractors/github.js

[hxrshxz]

Retrieving the last 30 days every time is not the way to go. Take a look at the other GitHub stats, we pull the data of the previous day and append it to a JSON, then commit to the repo. This means we have a growing permanent history. For the initial JSON you'd need to run the script for each day starting with the first day that ever had a spam PR/issue, then concatenate all subsequent days except the day that would be appended by the next CI run.

Also, the visualization is missing, and without the PR is too incomplete to merge.

Lastly, please remove your GitHub token. I think it works without - see the other GitHub stats again: https://github.com/juice-shop/juicy-statistics/blob/master/extractors/github.js

I will be occupied with my university exams until March 7th 😅 . After that I will go through the codebase to understand it better and implement a similar approach for the spam-report and make the necessary changes

[hxrshxz]

I am almost done with tihs , just need to figure out some stuff then i will push the changes

[hxrshxz]

For the initial JSON you'd need to run the script for each day starting with the first day that ever had a spam PR/issue,then concatenate all subsequent days except the day that would be appended by the next CI run.

Populated the spam-report.json with a script to fetch the no.of spam pr/issue starting from the 1st ever spam pr/issue

then concatenate all subsequent days except the day that would be appended by the next CI run.

added a similar function to achieve this after going throught the approach used in github.js.

Also, the visualization is missing, and without the PR is too incomplete to merge.

added two graphs (1st one is the default what we are using and another alternative(chart.js)) since the google chart's one doesn't seem as promising. Take a look at both and let me know which one we should proceed with. I will further improve the chosen graph to make it even better if possible

1st Graph

Screencast.from.2025-03-11.04-14-32.mp4

2nd Graph (chart.js)

Screencast.from.2025-03-11.04-04-55.mp4

[J12934]

sorry for chiming in there late.
just looking at the graphs, i think daily is way to fine as a granularity.
would be more interesting for me to see the monthly numbers to really understand how this has changed over time.
with the current graph this is pretty hard to tell how much this changed.

probably still fine to collect and store it on a daily basis, but wouldn't display it this way.

[bkimminich]

Collecting daily, displaying monthly sum sounds best, agreed.

[J12934]

For me the chart is taking up the entire screen width and height, which is really overwhelming, and doesn't really fit with the other charts. would limit the height to be more managable.

[J12934]

is there a reason this is pulling in another chart libary?
seems like this graph should also be achievable with the already used google visualization lib, don't want to pull in two libs unless we have a good reason for it

[hxrshxz]

yes true i have now used thee google lib

[hxrshxz]

Screencast.from.2025-03-18.17-27-45.mp4

[J12934]

looks good to me now, @bkimminich can you also confirm, as i haven't really touched the stats repo before?

[bkimminich]

Looks great! 👍 Thanks a lot, @hxrshxz!