Technology

Share Community
Report Community
11
Robots.txt for the NYT has a specific exclusion for an 1996 news article

Robots.txt for the NYT has a specific exclusion for an 1996 news article

3 years ago
Anonymous $hYN7Hy7o7J

https://twitter.com/oliviasolon/status/1171200746240372737

Robots.txt for the NYT has a specific exclusion for an 1996 news article

Mar 29, 2021, 3:17am UTC
$hYN7Hy7o7J
https://twitter.com/oliviasolon/status/1171200746240372737