googlebot and protector
List posts in the topic
googlebot and protector
msg# 1
Hi GiJ first my congrats for so cool module.
Second i want to ask a question, i had observed many "CRAWLER ATTACKS" on my installed protector module log, i took a look to the ips (all were 62.249.72.x) and realized this is googlebot, dor example the host for 66.249.72.140 is crawl-66-249-72-140.googlebot.com located in Mountain View (California).
I checked the allowed crawlers and have this in my settings /(msnbot|Googlebot|Yahoo! Slurp)/i
My question is.. the caps affect this? i mean its the same Googlebot and googlebot (real host name)? its protector module blocking googlebot crawler?
Ta in advance
Votes:0
Average:0.00
Posts tree