Enable script in HTTPWebRequest

[gorkos]

Hi,
I am two days trying to solve a problem with some pages, which i get through
HTTPWebRequest. Error is that some pages need Script to be enabled. But how
to do this in HTTPWebRequest class?

[Bruce Barker]

script is client code. when your code uses HTTPWebRequest, your code is the
client. the resest just returns the html, you need to write code to render
or run scripts. you could host the IE or mozilla active/x controls and use
them to render. neither are writen to be server based controls though.

-- bruce (sqlwork.com)
"gorkos" <go****@discussions.microsoft.com> wrote in message
news:C4**********************************@microsof t.com...

Hi,
I am two days trying to solve a problem with some pages, which i get
through
HTTPWebRequest. Error is that some pages need Script to be enabled. But
how
to do this in HTTPWebRequest class?

[Kevin Spencer]

The first step to solving a problem is to define the problem clearly and
completely. It is not possible to proceed with a solution until this first
step is completed.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

"gorkos" <go****@discussions.microsoft.com> wrote in message
news:C4**********************************@microsof t.com...

Hi,
I am two days trying to solve a problem with some pages, which i get
through
HTTPWebRequest. Error is that some pages need Script to be enabled. But
how
to do this in HTTPWebRequest class?

[gorkos]

Yes, thx for the lesson, exactly is something like this:

1. I need to download a page using HTTPWebRequest and than parse request;
2. Page requires to have Script enabled;
3. I suppose that script should be execute only after some user action (no
onLoad or else script).
4. So, for my purpose, for me is just enough if i can on someway simulate
than script is enabled, using HTTPWebRequest, to get full answer from site
(complete page)

"Kevin Spencer" wrote:

The first step to solving a problem is to define the problem clearly and
completely. It is not possible to proceed with a solution until this first
step is completed.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

"gorkos" <go****@discussions.microsoft.com> wrote in message
news:C4**********************************@microsof t.com...

Hi,
I am two days trying to solve a problem with some pages, which i get
through
HTTPWebRequest. Error is that some pages need Script to be enabled. But
how
to do this in HTTPWebRequest class?

[Kevin Spencer]

Hi gorkos,

I'm still having trouble understanding, but maybe it's a language barrier
thing. Let me try to ask a couple of questions.

1. You say you need to download a page using HTTPWebRequest. That's pretty
clear.

2. What do you mean by "Page requires to have Script enabled?" Apparently,
something is happening which seems to indicate this to you. What exactly is
it? What I mean is, an HTML document is a pretty straightforward text
document. All it contains is text. Any other resources in the page are
obtained via the browser making other requests, for images, other binary
resources, external scripts or stylesheets, etc. So, when you say "Page
requires to have Script enabled" I don't know what you're describing.

I get the feeling that you're trying to obtain something that is not IN the
HTML document you're requesting, but another resource that may be contained
in the page as a tag of some kind. But I can't tell for sure.

You also made some reference to some kind of "Error." Can you be more
specific about the nature of the "Error?"

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

"gorkos" <go****@discussions.microsoft.com> wrote in message
news:A6**********************************@microsof t.com...

Yes, thx for the lesson, exactly is something like this:

1. I need to download a page using HTTPWebRequest and than parse request;
2. Page requires to have Script enabled;
3. I suppose that script should be execute only after some user action (no
onLoad or else script).
4. So, for my purpose, for me is just enough if i can on someway simulate
than script is enabled, using HTTPWebRequest, to get full answer from site
(complete page)

"Kevin Spencer" wrote:

The first step to solving a problem is to define the problem clearly and
completely. It is not possible to proceed with a solution until this
first
step is completed.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

"gorkos" <go****@discussions.microsoft.com> wrote in message
news:C4**********************************@microsof t.com...

> Hi,
> I am two days trying to solve a problem with some pages, which i get
> through
> HTTPWebRequest. Error is that some pages need Script to be enabled. But
> how
> to do this in HTTPWebRequest class?

[Bruce Barker]

you need to figure out what the script does and emulate it. script can
change the UI, but for postback it generally only update the values of
postback fields (though its pretty easy to add postback fields in client
script, its not often done.)

you first need the parse the request response to get the form field names,
so you can build the postback name values pairs to emulate a submit. client
script normally only plays with the values of these fields.

for example to emulate a dropdown postback in asp.net (which uses client
script), you post the value of the selected value of the select, and
postback the id (after proper name munging) of the select in __EVENTTARGET
name pair.

-- bruce (sqlwork.com)

"gorkos" <go****@discussions.microsoft.com> wrote in message
news:A6**********************************@microsof t.com...

Yes, thx for the lesson, exactly is something like this:

1. I need to download a page using HTTPWebRequest and than parse request;
2. Page requires to have Script enabled;
3. I suppose that script should be execute only after some user action (no
onLoad or else script).
4. So, for my purpose, for me is just enough if i can on someway simulate
than script is enabled, using HTTPWebRequest, to get full answer from site
(complete page)

"Kevin Spencer" wrote:

The first step to solving a problem is to define the problem clearly and
completely. It is not possible to proceed with a solution until this
first
step is completed.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

"gorkos" <go****@discussions.microsoft.com> wrote in message
news:C4**********************************@microsof t.com...

> Hi,
> I am two days trying to solve a problem with some pages, which i get
> through
> HTTPWebRequest. Error is that some pages need Script to be enabled. But
> how
> to do this in HTTPWebRequest class?

[gorkos]

Hi,
OK, again. I will try to be more clear.
1. I download the page with HTTPWebRequest, and I expect right data, which i
have when i use a browser, to parse some data from the page.
2. Answer from the page is (without HTML tags):

Please activate scripting
Scripting must be enabled before you can continue.
To enable scripting, you need to make a simple adjustment to your browser
settings.

Click here if you need more help with your browser.
After you have enabled scripting refresh this page.

3. I want to avoid this message and to collect data which i expect (like in
a browser)

"Kevin Spencer" wrote:

Hi gorkos,

I'm still having trouble understanding, but maybe it's a language barrier
thing. Let me try to ask a couple of questions.

1. You say you need to download a page using HTTPWebRequest. That's pretty
clear.

2. What do you mean by "Page requires to have Script enabled?" Apparently,
something is happening which seems to indicate this to you. What exactly is
it? What I mean is, an HTML document is a pretty straightforward text
document. All it contains is text. Any other resources in the page are
obtained via the browser making other requests, for images, other binary
resources, external scripts or stylesheets, etc. So, when you say "Page
requires to have Script enabled" I don't know what you're describing.

I get the feeling that you're trying to obtain something that is not IN the
HTML document you're requesting, but another resource that may be contained
in the page as a tag of some kind. But I can't tell for sure.

You also made some reference to some kind of "Error." Can you be more
specific about the nature of the "Error?"

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

"gorkos" <go****@discussions.microsoft.com> wrote in message
news:A6**********************************@microsof t.com...

Yes, thx for the lesson, exactly is something like this:

1. I need to download a page using HTTPWebRequest and than parse request;
2. Page requires to have Script enabled;
3. I suppose that script should be execute only after some user action (no
onLoad or else script).
4. So, for my purpose, for me is just enough if i can on someway simulate
than script is enabled, using HTTPWebRequest, to get full answer from site
(complete page)

"Kevin Spencer" wrote:

The first step to solving a problem is to define the problem clearly and
completely. It is not possible to proceed with a solution until this
first
step is completed.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

"gorkos" <go****@discussions.microsoft.com> wrote in message
news:C4**********************************@microsof t.com...
> Hi,
> I am two days trying to solve a problem with some pages, which i get
> through
> HTTPWebRequest. Error is that some pages need Script to be enabled. But
> how
> to do this in HTTPWebRequest class?

[Kevin Spencer]

Hi gorkos,

What you posted is probably not the exact HTML in the page, as it does not
contain any tags. It would help if you posted the actual HTML you receive.
The reason is that there is probably a script in the page that writes out
the HTML you expect to see in a browser. You might also want to post the URL
to the page you are trying to parse, as this would help considerably. I
could look at the actual source code in the page and see your problem in a
heartbeat.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

"gorkos" <go****@discussions.microsoft.com> wrote in message
news:9E**********************************@microsof t.com...

Hi,
OK, again. I will try to be more clear.
1. I download the page with HTTPWebRequest, and I expect right data, which
i
have when i use a browser, to parse some data from the page.
2. Answer from the page is (without HTML tags):

Please activate scripting
Scripting must be enabled before you can continue.
To enable scripting, you need to make a simple adjustment to your browser
settings.

Click here if you need more help with your browser.
After you have enabled scripting refresh this page.

3. I want to avoid this message and to collect data which i expect (like
in
a browser)

"Kevin Spencer" wrote:

Hi gorkos,

I'm still having trouble understanding, but maybe it's a language barrier
thing. Let me try to ask a couple of questions.

1. You say you need to download a page using HTTPWebRequest. That's
pretty
clear.

2. What do you mean by "Page requires to have Script enabled?"
Apparently,
something is happening which seems to indicate this to you. What exactly
is
it? What I mean is, an HTML document is a pretty straightforward text
document. All it contains is text. Any other resources in the page are
obtained via the browser making other requests, for images, other binary
resources, external scripts or stylesheets, etc. So, when you say "Page
requires to have Script enabled" I don't know what you're describing.

I get the feeling that you're trying to obtain something that is not IN
the
HTML document you're requesting, but another resource that may be
contained
in the page as a tag of some kind. But I can't tell for sure.

You also made some reference to some kind of "Error." Can you be more
specific about the nature of the "Error?"

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

"gorkos" <go****@discussions.microsoft.com> wrote in message
news:A6**********************************@microsof t.com...

> Yes, thx for the lesson, exactly is something like this:
>
> 1. I need to download a page using HTTPWebRequest and than parse
> request;
> 2. Page requires to have Script enabled;
> 3. I suppose that script should be execute only after some user action
> (no
> onLoad or else script).
> 4. So, for my purpose, for me is just enough if i can on someway
> simulate
> than script is enabled, using HTTPWebRequest, to get full answer from
> site
> (complete page)
>
> "Kevin Spencer" wrote:
>
>> The first step to solving a problem is to define the problem clearly
>> and
>> completely. It is not possible to proceed with a solution until this
>> first
>> step is completed.
>>
>> --
>> HTH,
>>
>> Kevin Spencer
>> Microsoft MVP
>> ..Net Developer
>> Ambiguity has a certain quality to it.
>>
>> "gorkos" <go****@discussions.microsoft.com> wrote in message
>> news:C4**********************************@microsof t.com...
>> > Hi,
>> > I am two days trying to solve a problem with some pages, which i get
>> > through
>> > HTTPWebRequest. Error is that some pages need Script to be enabled.
>> > But
>> > how
>> > to do this in HTTPWebRequest class?
>>
>>
>>