Actually, robot traffic accounts for a much larger share of traffic for any of t... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

datawander on Feb 23, 2014 | parent | context | favorite | on: Google's robots.txt

Actually, robot traffic accounts for a much larger share of traffic for any of the big websites than you think.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact