469,926 Members | 2,288 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,926 developers. It's quick & easy.

fopen/fsockopen question

Dear All,

I want to write a simple PHP code to verify whether a given list of
URLs is broken or not. The URLs given have various formats, for example,

http://www.afro.com/history/history.html
http://www.worldhistorycompass.com/index.htm
http://www.afup.org/article.php3?id_article=242

fsockopen(...) only accepts target format (like www.yahoo.com etc), so I
simply choose fopen (url,'r') to do the test.

This method seems working well for majority of URLs I have. However,
when the following site seems causing problem,

$site = fopen("http://www.marketingpower.com/live/topics12.php",'r');

my browser seems working very hard on this link and after a long time,
the browser stops (not continuing on the next command. I am using
netscape on linux).

Initially I thought probably fopen doesn't like http://xxxx/xxx.php type
of format, however, links like:

http://www.afup.org/article.php3?id_article=242

works just fine.

So I am confused. Can anybody give me a hint on this?

Thanks.

Jul 17 '05 #1
5 2728
Shaoyong Wang wrote:
Dear All,

I want to write a simple PHP code to verify whether a given list of
URLs is broken or not. The URLs given have various formats, for example,

http://www.afro.com/history/history.html
http://www.worldhistorycompass.com/index.htm
http://www.afup.org/article.php3?id_article=242

fsockopen(...) only accepts target format (like www.yahoo.com etc), so I
simply choose fopen (url,'r') to do the test.

This method seems working well for majority of URLs I have. However,
when the following site seems causing problem,

$site = fopen("http://www.marketingpower.com/live/topics12.php",'r');

my browser seems working very hard on this link and after a long time,
the browser stops (not continuing on the next command. I am using
netscape on linux).

Initially I thought probably fopen doesn't like http://xxxx/xxx.php type
of format, however, links like:

http://www.afup.org/article.php3?id_article=242

works just fine.

So I am confused. Can anybody give me a hint on this?

Thanks.


I've tried to use nc to open the site you mentioned. It seems it does a
whole lot of redirecting and setting cookies. Might be that's what's
giving the URL wrapper trouble opening the page properly.

If that's the case, you will have no other choice than to write your own
HTTP request handler, which handles redirects and cookies. Which isn't
too hard, but it's a lot of work :).

-Rico
Jul 17 '05 #2
Thanks.

This link seems the only problematic one so far. I can just skip this one.
Notice fopen can't stop at this link, even I set max_execution_time in
php.ini, no idea.
I am using PHP 4.2.2

Jul 17 '05 #3
On Thu, 27 May 2004 13:11:59 +0200, Rico Huijbers
<E.************@REMOVEstudent.tue.nl> wrote:
Shaoyong Wang wrote:
Dear All,

I want to write a simple PHP code to verify whether a given list of
URLs is broken or not. The URLs given have various formats, for example,

http://www.afro.com/history/history.html
http://www.worldhistorycompass.com/index.htm
http://www.afup.org/article.php3?id_article=242

fsockopen(...) only accepts target format (like www.yahoo.com etc), so I
simply choose fopen (url,'r') to do the test.

This method seems working well for majority of URLs I have. However,
when the following site seems causing problem,

$site = fopen("http://www.marketingpower.com/live/topics12.php",'r');

my browser seems working very hard on this link and after a long time,
the browser stops (not continuing on the next command. I am using
netscape on linux).

Initially I thought probably fopen doesn't like http://xxxx/xxx.php type
of format, however, links like:

http://www.afup.org/article.php3?id_article=242

works just fine.

So I am confused. Can anybody give me a hint on this?

Thanks.


I've tried to use nc to open the site you mentioned. It seems it does a
whole lot of redirecting and setting cookies. Might be that's what's
giving the URL wrapper trouble opening the page properly.

If that's the case, you will have no other choice than to write your own
HTTP request handler, which handles redirects and cookies. Which isn't
too hard, but it's a lot of work :).


couldn't you just do something like this?:

$hostname = preg_replace('/http:\/\/([^\/]*).*/','$1',$url);
$conn_id = fsockopen('tcp://' . gethostbyname($hostname), 80,
$errno, $errstr);
fputs($conn_id,"GET $url HTTP/1.1\r\nHost: $hostname\r\n\r\n");

where $url is the url in question, and then where you do whatever you
would normally do?
Jul 17 '05 #4
Shaoyong Wang <wa*****@cis.ohio-state.edu> wrote in message news:<c9**********@news.cis.ohio-state.edu>...
Thanks.

This link seems the only problematic one so far. I can just skip this one.
Notice fopen can't stop at this link, even I set max_execution_time in
php.ini, no idea.
I am using PHP 4.2.2


1. If possible, use cURL functions instead. It is much handy
2. Check the user_agent setting of your ini file. Some sites may
lock weird/unknown user agents.

--
| Just another PHP saint |
Email: rrjanbiah-at-Y!com
Jul 17 '05 #5
Shaoyong Wang wrote:
Dear All,

I want to write a simple PHP code to verify whether a given list of
URLs is broken or not. The URLs given have various formats, for example,

http://www.afro.com/history/history.html
http://www.worldhistorycompass.com/index.htm
http://www.afup.org/article.php3?id_article=242

fsockopen(...) only accepts target format (like www.yahoo.com etc), so I
simply choose fopen (url,'r') to do the test.

This method seems working well for majority of URLs I have. However,
when the following site seems causing problem,

$site = fopen("http://www.marketingpower.com/live/topics12.php",'r');

my browser seems working very hard on this link and after a long time,
the browser stops (not continuing on the next command. I am using
netscape on linux).

Initially I thought probably fopen doesn't like http://xxxx/xxx.php type
of format, however, links like:

http://www.afup.org/article.php3?id_article=242

works just fine.

So I am confused. Can anybody give me a hint on this?

Thanks.


just use fopen

what you have to decide on are how many conditions do you have to deal with.

1) success!
2) a 404 response (file not found)
3) connection refused (port not listening)
4) timeout (server is down or not resolvable)

Jul 17 '05 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by theouimets | last post: by
3 posts views Thread by Ming | last post: by
12 posts views Thread by Bill | last post: by
2 posts views Thread by Loopy | last post: by
4 posts views Thread by peter.hrdy | last post: by
4 posts views Thread by Aetherweb | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.