Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Replace scala-uri library from ExtractDomain and just parse public_suffix_list.dat #521

Closed
ruebot opened this issue Oct 21, 2021 · 1 comment · Fixed by #524
Closed

Replace scala-uri library from ExtractDomain and just parse public_suffix_list.dat #521

ruebot opened this issue Oct 21, 2021 · 1 comment · Fixed by #524
Labels
clean-up enhancement

Comments

@ruebot
Copy link
Member

ruebot commented Oct 21, 2021

Since we're only using the library to grab domain apex, we can just roll our own implementation here.

Pull down public_suffix_list.dat, store it in memory, and do look-ups/matches based on it.

val publicSuffixes = scala.io.Source.fromURL("https://publicsuffix.org/list/public_suffix_list.dat", "utf-8").getLines.map(_.trim).filter(_.nonEmpty).filter(!_.startsWith("//")).toSet
@ruebot
Copy link
Member Author

ruebot commented Oct 21, 2021

Follow-up issue to #520

ruebot added a commit that referenced this issue Nov 1, 2021
@ruebot
Replace scala-uri library from ExtractDomain.
09ec587 
- Pull down public_suffix_list.dat, and parse it to ExtractDomain
  instead of using scala-uri
- Remove scala-uri
- Remove cats shading
- Resolves #521
@ruebot ruebot mentioned this issue Nov 1, 2021
Merged
ianmilligan1 pushed a commit that referenced this issue Nov 1, 2021
@ruebot
Replace scala-uri library from ExtractDomain. (#524)
f2b4379 
- Pull down public_suffix_list.dat, and parse it to ExtractDomain
  instead of using scala-uri
- Remove scala-uri
- Remove cats shading
- Resolves #521
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
clean-up enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Replace scala-uri library from ExtractDomain. archivesunleashed/aut
1 participant
@ruebot