CURL form posting to self referencing form action

I've successfully used CURL to maneuver around and through sites
but a new site I've been trying to work with has got me stumped.

To retrieve the data I'm trying to get a login is required. If
you go to any page in the "member" area without logging in you
get directed to a https connection and hit login.php (ie: you
end up at https://www.domain.com/login.php). The problem, I
believe, lies in the fact that the form on that page references
itself in the form action tag.

The action tag in the form is "/login.php" and nothing I've tried
has been successful in getting CURL to actually submit the data
and get me logged in. Not surprising really since you'd normally
point CURL to the action target to pass form data.

The things I've tried so far without luck are:

* Hit the main site page to pick up the session cookie then go
to one of the "member" only pages

* Gone directly to one of the "member" pages to get redirected
to login.php

Both attempts end up simply returning me the contents of the
login.php page which, obviously, doesn't help or do any good.

What isn't clear from the CURL documentation with PHP is whether
or not the CURLOPT_POSTFIELDS get passed to pages after a redirect
takes place. I suspect they are but the fact that login.php
refers back to itself seems to be the real problem.

If anyone has any ideas on how to convince CURL to get me past
login.php or any other suggestions on things I could try I would
very much appreciate it.

Thanks,
Dave Martin

re: CURL form posting to self referencing form action

What happens when you remove the "action" parameter from the <form> tag?
That makes the form re-submit to the exact same page...

"Dave Martin" <dave_martin@mindspring.com> wrote in message
news:MPG.1c1a1d02e07a16cc989683@news-out.barditch.com...[color=blue]
> I've successfully used CURL to maneuver around and through sites
> but a new site I've been trying to work with has got me stumped.
>
> To retrieve the data I'm trying to get a login is required. If
> you go to any page in the "member" area without logging in you
> get directed to a https connection and hit login.php (ie: you
> end up at https://www.domain.com/login.php). The problem, I
> believe, lies in the fact that the form on that page references
> itself in the form action tag.
>
> The action tag in the form is "/login.php" and nothing I've tried
> has been successful in getting CURL to actually submit the data
> and get me logged in. Not surprising really since you'd normally
> point CURL to the action target to pass form data.
>
> The things I've tried so far without luck are:
>
> * Hit the main site page to pick up the session cookie then go
> to one of the "member" only pages
>
> * Gone directly to one of the "member" pages to get redirected
> to login.php
>
> Both attempts end up simply returning me the contents of the
> login.php page which, obviously, doesn't help or do any good.
>
> What isn't clear from the CURL documentation with PHP is whether
> or not the CURLOPT_POSTFIELDS get passed to pages after a redirect
> takes place. I suspect they are but the fact that login.php
> refers back to itself seems to be the real problem.
>
> If anyone has any ideas on how to convince CURL to get me past
> login.php or any other suggestions on things I could try I would
> very much appreciate it.
>
> Thanks,
> Dave Martin[/color]

re: CURL form posting to self referencing form action

> What happens when you remove the "action" parameter from the <form> tag? That makes the form re-submit to the exact same page...

Nope. It makes the browser responsible for handling such form. Maybe
some browsers do it like you said, but they may change this behaviour
in any new wersion without warning.
Better way to achieve the same thing is to use $_SERVER['PHP_SELF']
or something like that as "action" attribute value in "form" tag.

Hilarion

re: CURL form posting to self referencing form action

.oO(d)
[color=blue]
>What happens when you remove the "action" parameter from the <form> tag?[/color]

The action-attribute is required. Omitting it is invalid code.
[color=blue]
>That makes the form re-submit to the exact same page...[/color]

Not necessarily.

Micha

re: CURL form posting to self referencing form action

Couple of suggestions:

If the site you are trying to access requires sessions/cookies, are you sure
you're capturing these correctly? Try to get your script to successfully
login and capture the cookies to a cookie jar. This is probably where the
problem lies.

If you can't figure out what your browser is doing that your script isn't,
use a proxy debugger like Charles 1.8. This lets you capture exactly what
the browser sends and gets back.

Hopefully helpful,
Kevin

"Dave Martin" <dave_martin@mindspring.com> wrote in message
news:MPG.1c1a1d02e07a16cc989683@news-out.barditch.com...[color=blue]
> I've successfully used CURL to maneuver around and through sites
> but a new site I've been trying to work with has got me stumped.
>
> To retrieve the data I'm trying to get a login is required. If
> you go to any page in the "member" area without logging in you
> get directed to a https connection and hit login.php (ie: you
> end up at https://www.domain.com/login.php). The problem, I
> believe, lies in the fact that the form on that page references
> itself in the form action tag.
>
> The action tag in the form is "/login.php" and nothing I've tried
> has been successful in getting CURL to actually submit the data
> and get me logged in. Not surprising really since you'd normally
> point CURL to the action target to pass form data.
>
> The things I've tried so far without luck are:
>
> * Hit the main site page to pick up the session cookie then go
> to one of the "member" only pages
>
> * Gone directly to one of the "member" pages to get redirected
> to login.php
>
> Both attempts end up simply returning me the contents of the
> login.php page which, obviously, doesn't help or do any good.
>
> What isn't clear from the CURL documentation with PHP is whether
> or not the CURLOPT_POSTFIELDS get passed to pages after a redirect
> takes place. I suspect they are but the fact that login.php
> refers back to itself seems to be the real problem.
>
> If anyone has any ideas on how to convince CURL to get me past
> login.php or any other suggestions on things I could try I would
> very much appreciate it.
>
> Thanks,
> Dave Martin[/color]

re: CURL form posting to self referencing form action

"d" <d@example.com> wrote in message
news:<ZJYrd.69744$F7.32173@fe1.news.blueyonder.co. uk>...[color=blue]
>
> What happens when you remove the "action" parameter from the <form> tag?
> That makes the form re-submit to the exact same page...[/color]

Nope. It makes the form submit to the default file in the current
directory.

"Dave Martin" <dave_martin@mindspring.com> wrote in message
news:MPG.1c1a1d02e07a16cc989683@news-out.barditch.com...[color=blue]
>
> To retrieve the data I'm trying to get a login is required. If
> you go to any page in the "member" area without logging in you
> get directed to a https connection and hit login.php (ie: you
> end up at https://www.domain.com/login.php). The problem, I
> believe, lies in the fact that the form on that page references
> itself in the form action tag.
>
> The action tag in the form is "/login.php" and nothing I've tried
> has been successful in getting CURL to actually submit the data
> and get me logged in. Not surprising really since you'd normally
> point CURL to the action target to pass form data.
>
> The things I've tried so far without luck are:
>
> * Hit the main site page to pick up the session cookie then go
> to one of the "member" only pages
>
> * Gone directly to one of the "member" pages to get redirected
> to login.php[/color]

There is no silver bullet here. You need to study the server's
raw output at each step of the authentication process. Keep in
mind there may be redirects involved, so you need to follow them
while keeping all relevant cookies...

Cheers,
NC

re: CURL form posting to self referencing form action

Hello,

On 12/03/2004 09:50 AM, Dave Martin wrote:[color=blue]
> The action tag in the form is "/login.php" and nothing I've tried
> has been successful in getting CURL to actually submit the data
> and get me logged in. Not surprising really since you'd normally
> point CURL to the action target to pass form data.
>
> The things I've tried so far without luck are:
>
> * Hit the main site page to pick up the session cookie then go
> to one of the "member" only pages
>
> * Gone directly to one of the "member" pages to get redirected
> to login.php
>
> Both attempts end up simply returning me the contents of the
> login.php page which, obviously, doesn't help or do any good.[/color]

This is odd because Curl is supposed to do collect any cookies.

Personally I only use curl for accessing SSL pages. For non-SSL pages I
just use TCP socket connections.

Anyway, even when I use curl, I use this HTTP client class that compose
the request headers and bodies itself instead of relying on curl
abilities to compose the kind of posts that I want and that works
wonders supporting certain types of things that I am not sure curl supports.

You may also want to try this HTTP client yourself and let me know if
you still have any difficulties. It is also capable of collecting
cookies and handling redirection which is something you also need. It
comes with examples to show that:

http://www.phpclasses.org/httpclient

--

Regards,
Manuel Lemos

PHP Classes - Free ready to use OOP components written in PHP
http://www.phpclasses.org/

PHP Reviews - Reviews of PHP books and other products
http://www.phpclasses.org/reviews/

Metastorage - Data object relational mapping layer generator
http://www.meta-language.net/metastorage.html

re: CURL form posting to self referencing form action

> "Dave Martin" <dave_martin@mindspring.com> wrote in message[color=blue]
> news:MPG.1c1a1d02e07a16cc989683@news-out.barditch.com...[/color]
<snip>
[color=blue][color=green]
> > The things I've tried so far without luck are:
> >
> > * Hit the main site page to pick up the session cookie then go
> > to one of the "member" only pages[/color][/color]

How, did you do that? Did you instruct cURL to follow cookie like
this <http://curl.haxx.se/libcurl/php/examples/?ex=cookiejar.php>?

<snip>[color=blue][color=green]
> > What isn't clear from the CURL documentation with PHP is whether
> > or not the CURLOPT_POSTFIELDS get passed to pages after a redirect
> > takes place. I suspect they are but the fact that login.php
> > refers back to itself seems to be the real problem.[/color][/color]

Perhaps, the site does some REFERER check--but you didn't send it??
[color=blue][color=green]
> > If anyone has any ideas on how to convince CURL to get me past
> > login.php or any other suggestions on things I could try I would
> > very much appreciate it.[/color][/color]

There are lot of cURL variables <http://in.php.net/curl> You may
want to set CURLOPT_FOLLOWLOCATION

--
<?php echo 'Just another PHP saint'; ?>
Email: rrjanbiah-at-Y!com Blog: http://rajeshanbiah.blogspot.com/

re: CURL form posting to self referencing form action

In article <abc4d8b8.0412040855.3bdc9c1c@posting.google.com >,
ng4rrjanbiah@rediffmail.com says...
[color=blue]
> How, did you do that? Did you instruct cURL to follow cookie like
> this <http://curl.haxx.se/libcurl/php/examples/?ex=cookiejar.php>?[/color]

Nope and that was exactly my problem. I had been trying to grab the one
cookie I knew I needed but apparently that wasn't working (at least not
correctly as I know I was getting cookie data). Wished I'd found that
page a LOT sooner :)
[color=blue]
> There are lot of cURL variables <http://in.php.net/curl> You may
> want to set CURLOPT_FOLLOWLOCATION[/color]

Already had it enabled and I was actually following the redirects. That
much I could see in the CURL trace.

Dave

re: CURL form posting to self referencing form action

dear sir,

not entierly sure of what the solution is, but i had a similar experience
with google. i tried to connect to google manually using http connection to
preform a search. i however could not get to the second page. the reason
being that there was a param 'ie=UTF' also required along with other
variables. i searched the original form, i could not find the variable 'ie'.
but i saw a javascript within the form. google has slyly include the extra
param within the form.

my suggestion,
1. search for extra param, if any
2. check if your script preforms a SSL connection correctly.

i hope this helps.

sumeet shroff


"Dave Martin" <dave_martin@mindspring.com> wrote in message
news:MPG.1c1a1d02e07a16cc989683@news-out.barditch.com...[color=blue]
> I've successfully used CURL to maneuver around and through sites
> but a new site I've been trying to work with has got me stumped.
>
> The action tag in the form is "/login.php" and nothing I've tried
> has been successful in getting CURL to actually submit the data
> and get me logged in. Not surprising really since you'd normally
> point CURL to the action target to pass form data.
>
>
> If anyone has any ideas on how to convince CURL to get me past
> login.php or any other suggestions on things I could try I would
> very much appreciate it.
>
> Thanks,
> Dave Martin[/color]