By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
445,857 Members | 1,768 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 445,857 IT Pros & Developers. It's quick & easy.

googlebot and CheckVirtualFileExists Exceptions!

P: n/a
Hi everyone!

I get some errors lately regarding:
HTTP_USER_AGENT Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)

and:

System.Web.UI.Util.CheckVirtualFileExists(VirtualP ath virtualPath)

After googling that,
I found this page:
http://www.inventua.com/forums.conte...948&view=topic

which notes the following:

The user-agent for this one is the Googlebot, which is known to deliberately
generate bad Urls sometimes as part of it's algorithm.

It will keep happening when the GoogleBot tries to index your page. It's
"remembering" the existance of the page from the last time it indexed your
site.

I'm not real familiar with the GoogleBot, but it sounds like that's Google's
bot that crawls around the internet indexing pages. I read an article on the
DotNetNuke site that talked about GoogleBot generated errors taxing the
webserver, and eventually causing an IIS restart due to memory issues. The
general exception errors build up in memory until they are batched to the DB,
and if you get enough errors quick enough, IIS thinks there's a memory leak
and restarts. So, is there anythign I can do about this?
So, does anyone know of a way we can stop googlebot from generating these
errors?

Thanks in advance!
Jul 12 '07 #1
Share this Question
Share on Google+
5 Replies


P: n/a
re:
!So, does anyone know of a way we can stop googlebot from generating these errors?

Redirect 404's to a page of your own choosing.


Juan T. Llibre, asp.net MVP
asp.net faq : http://asp.net.do/faq/
foros de asp.net, en español : http://asp.net.do/foros/
======================================
"patrickdrd" <pa********@discussions.microsoft.comwrote in message
news:AD**********************************@microsof t.com...
Hi everyone!

I get some errors lately regarding:
HTTP_USER_AGENT Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)

and:

System.Web.UI.Util.CheckVirtualFileExists(VirtualP ath virtualPath)

After googling that,
I found this page:
http://www.inventua.com/forums.conte...948&view=topic

which notes the following:

The user-agent for this one is the Googlebot, which is known to deliberately
generate bad Urls sometimes as part of it's algorithm.

It will keep happening when the GoogleBot tries to index your page. It's
"remembering" the existance of the page from the last time it indexed your
site.

I'm not real familiar with the GoogleBot, but it sounds like that's Google's
bot that crawls around the internet indexing pages. I read an article on the
DotNetNuke site that talked about GoogleBot generated errors taxing the
webserver, and eventually causing an IIS restart due to memory issues. The
general exception errors build up in memory until they are batched to the DB,
and if you get enough errors quick enough, IIS thinks there's a memory leak
and restarts. So, is there anythign I can do about this?

So, does anyone know of a way we can stop googlebot from generating these
errors?

Thanks in advance!

Jul 12 '07 #2

P: n/a
Redirect 404's to a page of your own choosing.

I don't want that,
I need to somehow stop googlebot from trying to hit these 'invalid' urls,
is there any workaround for this?
Jul 12 '07 #3

P: n/a
re:
!>Redirect 404's to a page of your own choosing.
!I don't want that

That's the best solution.

It's not expensive, in terms of CPU cycles, and it eliminates
the memory problems stemming from unresolved urls.

re:
!I need to somehow stop googlebot from trying to hit these 'invalid' urls

You can't do that. Don't bang your head against the wall.
You cannot stop queries to invalid urls...no matter who sends them.

re:
!is there any workaround for this?

Not that I know of, but don't let me stop you.

Try to find a different solution.

Juan T. Llibre, asp.net MVP
asp.net faq : http://asp.net.do/faq/
foros de asp.net, en español : http://asp.net.do/foros/
======================================
"patrickdrd" <pa********@discussions.microsoft.comwrote in message
news:01**********************************@microsof t.com...
>Redirect 404's to a page of your own choosing.

I don't want that,
I need to somehow stop googlebot from trying to hit these 'invalid' urls,
is there any workaround for this?

Jul 12 '07 #4

P: n/a


"Juan T. Llibre" wrote:
re:
!>Redirect 404's to a page of your own choosing.
!I don't want that

That's the best solution.

It's not expensive, in terms of CPU cycles, and it eliminates
the memory problems stemming from unresolved urls.

re:
!I need to somehow stop googlebot from trying to hit these 'invalid' urls

You can't do that. Don't bang your head against the wall.
You cannot stop queries to invalid urls...no matter who sends them.

re:
!is there any workaround for this?

Not that I know of, but don't let me stop you.

Try to find a different solution.

Juan T. Llibre, asp.net MVP
asp.net faq : http://asp.net.do/faq/
foros de asp.net, en español : http://asp.net.do/foros/
======================================
"patrickdrd" <pa********@discussions.microsoft.comwrote in message
news:01**********************************@microsof t.com...
Redirect 404's to a page of your own choosing.
I don't want that,
I need to somehow stop googlebot from trying to hit these 'invalid' urls,
is there any workaround for this?


ok, thanks for the answer,
some other thing,
is there any way to find out why does googlebot hit those invalid urls?
Jul 12 '07 #5

P: n/a
re:
!is there any way to find out why does googlebot hit those invalid urls?

You might want to sieve through this Google query :
http://www.google.com/search?q=googl...&start=10&sa=N

This article has some tips :

http://www.seo4fun.com/notes/supplementals.html

Juan T. Llibre, asp.net MVP
asp.net faq : http://asp.net.do/faq/
foros de asp.net, en español : http://asp.net.do/foros/
======================================
"patrickdrd" <pa********@discussions.microsoft.comwrote in message
news:7C**********************************@microsof t.com...
>

"Juan T. Llibre" wrote:
>re:
!>Redirect 404's to a page of your own choosing.
!I don't want that

That's the best solution.

It's not expensive, in terms of CPU cycles, and it eliminates
the memory problems stemming from unresolved urls.

re:
!I need to somehow stop googlebot from trying to hit these 'invalid' urls

You can't do that. Don't bang your head against the wall.
You cannot stop queries to invalid urls...no matter who sends them.

re:
!is there any workaround for this?

Not that I know of, but don't let me stop you.

Try to find a different solution.

Juan T. Llibre, asp.net MVP
asp.net faq : http://asp.net.do/faq/
foros de asp.net, en español : http://asp.net.do/foros/
======================================
"patrickdrd" <pa********@discussions.microsoft.comwrote in message
news:01**********************************@microso ft.com...
>Redirect 404's to a page of your own choosing.

I don't want that,
I need to somehow stop googlebot from trying to hit these 'invalid' urls,
is there any workaround for this?



ok, thanks for the answer,
some other thing,
is there any way to find out why does googlebot hit those invalid urls?

Jul 12 '07 #6

This discussion thread is closed

Replies have been disabled for this discussion.