I RUM A VERY SMALL FORUM, INFORME.COM HOST MY FORUM SOMETHING TO DO WITH phpbb, its free.
i have notices on my whos on line section that i am getting a lot of visits by ip numbers belonging to inktomi.corp in california, it shows up on my forum as guests visits, it looks like its guest reading posts or even sending messages ect, but there is no one there, i presume they (inktomi) are crawling my site for info or something, i have been in touch with informe who basically say the same, is there anything i can do to stop the visits as it is a false record of visits to my forum or maybe worse.
1)you can block the IP addresses in your web server
2)inktomi is a respectable company… so i am assuming they will respect what says on your robot.txt on your website. on this robot.txt you can instruct the crawler not to crawl your site… (lots of info as to what to write into robot.txt on the web).