Maverick wrote:
How can I regulate an agent or bot to my site monthly ?
they are visiting to often
I'd like to allow them monthly only
any ideas ?
You could try generateing robots.txt dynamically. Based on the user
agent, just decide whether you wish to let it in today or not, and
based on that decision, output the user agent deny for it or don't.
It's good to know however that not all robots follow the robots.txt
(though the well behaving ones do). Another thing to note, is that
Google (for example) drops the pages it can't access from it's
database.
--
Markku Uttula