Stop HttpWebRequest on my site

Hello,

Someone is using HttpWebRequest to automaticly post datas or retrieve
datas from my site.
How can i stop pages to be get by the HttpWebRequest method ?

I know that pages get with HttpWebRequest have is no referer.
So for me the solution is to check the referer of the calling page, if
the referer is my domain, ok i can send the page, if not i can redirect
to an error page.
Is it a good idea ?
If you have other solution ?

Thanks

re: Stop HttpWebRequest on my site

you can add a referer header to a HttpWebRequest. there is really no way to
detect someone useing HttpWebRequest (or like) from a browser. also many
proxy servers strip the referer header, so you may block valid browser
users.

-- bruce (sqlwork.com)



"Not4u" <Not4u@chez.com> wrote in message
news:4420118d$0$7769$636a55ce@news.free.fr...[color=blue]
> Hello,
>
> Someone is using HttpWebRequest to automaticly post datas or retrieve
> datas from my site.
> How can i stop pages to be get by the HttpWebRequest method ?
>
> I know that pages get with HttpWebRequest have is no referer.
> So for me the solution is to check the referer of the calling page, if the
> referer is my domain, ok i can send the page, if not i can redirect to an
> error page.
> Is it a good idea ?
> If you have other solution ?
>
> Thanks
>[/color]

re: Stop HttpWebRequest on my site

So there is no solution to block the use of HttpWebRequest ?.


Bruce Barker wrote:[color=blue]
> you can add a referer header to a HttpWebRequest. there is really no way to
> detect someone useing HttpWebRequest (or like) from a browser. also many
> proxy servers strip the referer header, so you may block valid browser
> users.
>
> -- bruce (sqlwork.com)
>
>
>
> "Not4u" <Not4u@chez.com> wrote in message
> news:4420118d$0$7769$636a55ce@news.free.fr...[color=green]
>> Hello,
>>
>> Someone is using HttpWebRequest to automaticly post datas or retrieve
>> datas from my site.
>> How can i stop pages to be get by the HttpWebRequest method ?
>>
>> I know that pages get with HttpWebRequest have is no referer.
>> So for me the solution is to check the referer of the calling page, if the
>> referer is my domain, ok i can send the page, if not i can redirect to an
>> error page.
>> Is it a good idea ?
>> If you have other solution ?
>>
>> Thanks
>>[/color]
>
>[/color]

re: Stop HttpWebRequest on my site

For posting data to your site, you probably would have to implement
some sort of human verification scheme to detect that a actual person
is actually doing the posting and not a scrip of some sort. You see
this all the time where website display a distored image with letters
and numbers and you have to type what is displayed in the image to
continue. This will trap scrips from posting data.

For reading, you probably have to protect those pages you don't want
them to read and require the person wanting to read to login first.

re: Stop HttpWebRequest on my site

Yes i'll use the solution of random images.
My problem is it's a price comparison bot that grab info from my site
and use it to compare prices.

tdavisjr wrote:[color=blue]
> For posting data to your site, you probably would have to implement
> some sort of human verification scheme to detect that a actual person
> is actually doing the posting and not a scrip of some sort. You see
> this all the time where website display a distored image with letters
> and numbers and you have to type what is displayed in the image to
> continue. This will trap scrips from posting data.
>
> For reading, you probably have to protect those pages you don't want
> them to read and require the person wanting to read to login first.
>[/color]

re: Stop HttpWebRequest on my site

Thus wrote Not4u,
[color=blue]
> Hello,
>
> Someone is using HttpWebRequest to automaticly post datas or retrieve
> datas from my site.
> How can i stop pages to be get by the HttpWebRequest method ?
> I know that pages get with HttpWebRequest have is no referer.[/color]

That's not true. There's no way of knowing whether a request was sent by
a browser or some application simply sending the very same HTTP headers (unless
you could tell from certain implementation specific behaviors at TCP level...).

[color=blue]
> So for me the solution is to check the referer of the calling page, if
> the referer is my domain, ok i can send the page, if not i can
> redirect
> to an error page.
> Is it a good idea ?
> If you have other solution ?[/color]

If you think that you're victimized by a robot, you may try to add a robots.txt
(see http://www.robotstxt.org/wc/faq.html#robotstxt) to your web site.

Another way is to look for suspicious access patterns in your web server
logs that belong to certain source IP addresses, and block these addresses
at a firewall.

Cheers,
--
Joerg Jooss
news-reply@joergjooss.de