Security

5433 readers

3 users here now

Confidentiality Integrity Availability

founded 5 years ago

MODERATORS

497

submitted 1 year ago by [email protected] to c/[email protected]

63 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 7 points 1 year ago

robots.txt doesn't have to be followed. It doesn't block crawling.