this post was submitted on 07 Apr 2024
497 points (95.6% liked)
Security
5041 readers
6 users here now
Confidentiality Integrity Availability
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
There's also the question of what happens if they just ignore the robots.txt file