Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

robots.txt rework #504

Closed
quantum5 opened this issue Dec 16, 2016 · 1 comment · Fixed by #1427
Closed

robots.txt rework #504

quantum5 opened this issue Dec 16, 2016 · 1 comment · Fixed by #1427
Labels
enhancement
Milestone
Holidays 2016

Comments

@quantum5
Copy link
Member

We have not touched that file, despite adding new URLs for the past two years. Something tells me it's not relevant anymore...
@quantum5 quantum5 modified the milestone: Holidays 2016 Dec 21, 2016
@kiritofeng
Copy link
Member

API v2 should also be excluded from robots.txt

kiritofeng referenced this issue Feb 19, 2017
Prevent search engines from crawlling tickets
fdb92b4 
Search engines wouldn't be able to see it I think
Ninjaclasher added a commit to Ninjaclasher/site that referenced this issue Jun 4, 2020
@Ninjaclasher
Update robots.txt with new URL endpoints; DMOJ#504
5c5aaf8
Ninjaclasher added a commit to Ninjaclasher/site that referenced this issue Jun 4, 2020
@Ninjaclasher
Update robots.txt with new URL endpoints; DMOJ#504
c7c6dd6
Ninjaclasher added a commit to Ninjaclasher/site that referenced this issue Jun 4, 2020
@Ninjaclasher
Update robots.txt with new URL endpoints; DMOJ#504
8ffd6ec
@Ninjaclasher Ninjaclasher mentioned this issue Jun 4, 2020
Merged
Ninjaclasher added a commit to Ninjaclasher/site that referenced this issue Jun 6, 2020
@Ninjaclasher
Update robots.txt with new URL endpoints; DMOJ#504
bf04b1f
Xyene pushed a commit that referenced this issue Jun 6, 2020
@Ninjaclasher @Xyene
Update robots.txt with new URL endpoints; #504
8ed03b2
Ninjaclasher added a commit to Ninjaclasher/site that referenced this issue Jun 7, 2020
@Ninjaclasher
Update robots.txt with new URL endpoints; DMOJ#504
a714d21
outloudvi pushed a commit to SchOJ/site that referenced this issue Oct 27, 2020
@Ninjaclasher @outloudvi
Update robots.txt with new URL endpoints; DMOJ#504
22fae11
malbareda pushed a commit to malbareda/JOEL-web that referenced this issue Jan 4, 2024
@Ninjaclasher @Xyene
Update robots.txt with new URL endpoints; DMOJ#504
e362fe2
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
Holidays 2016
Development

Successfully merging a pull request may close this issue.

Update robots.txt with new URL endpoints; #504 Ninjaclasher/site
2 participants
@quantum5 @kiritofeng