Re: Ban IP when accessing a web page

List posts in the topic

Re: Ban IP when accessing a web page

msg# 1.1.1.1

depth:

3

Previous post - Next post | Parent - Children.1 .2 | Posted on 2006/8/4 6:07

GIJOE    Posts: 4110

Oh, you are right.

You have to add the file's path into robots.txt

- Good crawlers like googlebot
-- read robots.txt first
-- they don't access this trap file
-- your site will be indexed correctly

- Bad crawlers
-- ignore robots.txt
-- they access the trap file
-- the crawler is banned!

Votes:11 Average:8.18

Posts tree

• Ban IP when accessing a web page (danand725, 2006/8/1 19:10)
  • Re: Ban IP when accessing a web page (GIJOE, 2006/8/2 5:38)
    • Re: Ban IP when accessing a web page (danand725, 2006/8/2 18:36)
      • Re: Ban IP when accessing a web page (GIJOE, 2006/8/4 6:07)
        • Re: Ban IP when accessing a web page (danand725, 2006/8/9 19:37)
        • Re: Ban IP when accessing a web page (danand725, 2006/8/9 20:09)
          • Re: Ban IP when accessing a web page (GIJOE, 2006/8/10 6:10)
            • Re: Ban IP when accessing a web page (danand725, 2006/8/10 19:29)