fopen() unable to open some URLs?

[Michael Ferrier]

Hi,

I've used fopen() extensively to open web pages. I've found that there is a
small minority
of web pages that open fine in a browser, but are inaccessible using
fopen(). Here are
two such URLs:

http://www.homes.com/
http://www.dolbyproperties.com/

For example, using the code below, almost every web page that will open in a
browser
will be read correctly by this code. There are few exceptions, such as the
above URLs.
I'm wondering why/how could a page be accessible to a browser but not to
fopen()?

Here's the code:

<?php

$page = "";

if (isset($url))
{
$fp = fopen($url, 'r');

if ($fp != false)
{
while (feof($fp) == false)
{
$page .= fread($fp, 10000);
}

fclose($fp);

echo "URL: $url =============== =============== =======<br><br> ";
echo $page;
}
}

?>

I have this code running with a simple form interface here:

http://linkmachine.net/fopen_test.php

Any ideas?

Thank you,

-Michael

[Janwillem Borleffs]

Michael Ferrier wrote:

For example, using the code below, almost every web page that will
open in a browser
will be read correctly by this code. There are few exceptions, such
as the above URLs.
I'm wondering why/how could a page be accessible to a browser but not
to fopen()?

Some websites require an explicit user-agent header in order to return a
response, as is the case with homes.com

There are several ways to include the user agent, of which the easiest is to
apply the ini_set function as follows:

ini_set('user_a gent','Mozilla' );
$fp = fopen('http://www.homes.com/','r');
fpassthru($fp);

When you run this code, you will see it works fine for homes.com. Bare in
mind that this doesn't work for sites which require more then a user agent
string to operate.

JW