Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Support Shadowserver Vulnerable SMTP server feed, fix #1984 #2037

Merged
6 commits merged into from
Aug 13, 2021

Conversation

monoidic
Copy link
Contributor

@monoidic monoidic commented Aug 12, 2021

This PR adds support for the Shadowserver Vulnerable SMTP Report and fixes #1984 by differentiating between Vulnerable-HTTP and Accessible-HTTP filenames.

@codecov-commenter
Copy link

codecov-commenter commented Aug 12, 2021

Codecov Report

Merging #2037 (4ab22f7) into develop (7eaf71e) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop    #2037      +/-   ##
===========================================
- Coverage    75.98%   75.95%   -0.04%     
===========================================
  Files          423      427       +4     
  Lines        22812    22973     +161     
  Branches      3040     3060      +20     
===========================================
+ Hits         17334    17448     +114     
- Misses        4772     4817      +45     
- Partials       706      708       +2     
Impacted Files Coverage Δ
intelmq/bots/parsers/shadowserver/_config.py 97.27% <100.00%> (+0.01%) ⬆️
...lmq/tests/bots/parsers/shadowserver/test_broken.py 100.00% <100.00%> (ø)
.../parsers/shadowserver/test_scan_http_vulnerable.py 100.00% <100.00%> (ø)
.../parsers/shadowserver/test_scan_smtp_vulnerable.py 100.00% <100.00%> (ø)
intelmq/lib/upgrades.py 69.66% <0.00%> (ø)
...lmq/tests/bots/experts/domain_valid/test_expert.py 100.00% <0.00%> (ø)
intelmq/bots/experts/domain_valid/expert.py 42.85% <0.00%> (ø)
intelmq/bots/experts/ripe/expert.py 77.55% <0.00%> (+1.02%) ⬆️

@ghost ghost added this to the 3.0.1 milestone Aug 13, 2021
@ghost ghost added bug Indicates an unexpected problem or unintended behavior component: bots labels Aug 13, 2021
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add an entry in docs/user/bots.rst in the shadowserver feed section? Thanks

otherwise ready for merge

@@ -2862,8 +2888,9 @@ def scan_exchange_identifier(field):
('Sinkhole-Events-HTTP-Referer IPv6', 'event6_sinkhole_http_referer', event46_sinkhole_http_referer),
('Spam-URL', 'spam_url', spam_url),
('Vulnerable-ISAKMP', 'scan_isakmp', vulnerable_isakmp),
('Vulnerable-HTTP', 'scan_http', accessible_vulnerable_http),
('Vulnerable-HTTP', 'scan_http_vulnerable', accessible_vulnerable_http),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks!

@monoidic
Copy link
Contributor Author

monoidic commented Aug 13, 2021

The suggested improvement reminded me of #1984, so I also fixed that by making Vulnerable-HTTP match by scan_http_vulnerable rather than scan_http, and added some standalone tests for Vulnerable-HTTP (previously, only Accessible-HTTP had tests).

@monoidic monoidic changed the title Support Shadowserver Vulnerable SMTP server feed Support Shadowserver Vulnerable SMTP server feed, fix #1984 Aug 13, 2021
@ghost
Copy link

ghost commented Aug 13, 2021

The suggested improvement reminded me of #1984,

yeah, me too - but only after you committed the other fix :)

so I also fixed that by making Vulnerable-HTTP match by scan_http_vulnerable rather than scan_http, and added some standalone tests for Vulnerable-HTTP (previously, only Accessible-HTTP had tests).

Thanks! Didn't think that #1984 is based on such a simple mistake.

@ghost ghost merged commit 103a584 into certtools:develop Aug 13, 2021
@ghost
Copy link

ghost commented Aug 13, 2021

Thanks! Also cherry-picked for maintenance in 7a81e37 plus changelog in b76520e

This pull request was closed.
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Indicates an unexpected problem or unintended behavior component: bots
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Shadowserver Parser can't differentiate between "Accessible HTTP" and "Vulnerable HTTP"
2 participants