473,326 Members | 2,134 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

fopen() unable to open some URLs?

Hi,

I've used fopen() extensively to open web pages. I've found that there is a
small minority
of web pages that open fine in a browser, but are inaccessible using
fopen(). Here are
two such URLs:

http://www.homes.com/
http://www.dolbyproperties.com/

For example, using the code below, almost every web page that will open in a
browser
will be read correctly by this code. There are few exceptions, such as the
above URLs.
I'm wondering why/how could a page be accessible to a browser but not to
fopen()?

Here's the code:

<?php

$page = "";

if (isset($url))
{
$fp = fopen($url, 'r');

if ($fp != false)
{
while (feof($fp) == false)
{
$page .= fread($fp, 10000);
}

fclose($fp);

echo "URL: $url =====================================<br><br>";
echo $page;
}
}

?>

I have this code running with a simple form interface here:

http://linkmachine.net/fopen_test.php

Any ideas?

Thank you,

-Michael


Aug 11 '05 #1
1 2343
Michael Ferrier wrote:
For example, using the code below, almost every web page that will
open in a browser
will be read correctly by this code. There are few exceptions, such
as the above URLs.
I'm wondering why/how could a page be accessible to a browser but not
to fopen()?


Some websites require an explicit user-agent header in order to return a
response, as is the case with homes.com

There are several ways to include the user agent, of which the easiest is to
apply the ini_set function as follows:

ini_set('user_agent','Mozilla');
$fp = fopen('http://www.homes.com/','r');
fpassthru($fp);

When you run this code, you will see it works fine for homes.com. Bare in
mind that this doesn't work for sites which require more then a user agent
string to operate.

JW

Aug 11 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Shaoyong Wang | last post by:
Dear All, I want to write a simple PHP code to verify whether a given list of URLs is broken or not. The URLs given have various formats, for example, http://www.afro.com/history/history.html...
3
by: Ming | last post by:
When I use fopen on the URL: fopen("http://home.pchome.com.tw/world/qoo4ko/110.jpg";, "r") I would get the following error message: -----------Error Message------------ Warning: fopen():...
10
by: Longfellow | last post by:
Newbie here... My reading of the description of fopen() led me to expect that, with mode as "w", it would create a file if it did not exist. Checked the FAQ and did not see this question...
5
by: eyalc1978 | last post by:
Hi Does someone knows if fopen() is a non-blocking function in a sense that if there isn't sufficient disk space to open a file in write mode, then the function will not wait but fail? I find...
4
by: lucyachammond | last post by:
I have a 3rd party real-estate website written in php. I would like to call an asp script on a separate webserver each time a new real-estate item is added to the database on the php site. The...
1
by: nebelraabe | last post by:
hi there. I encountered a problem when i wanted to open a file which name i read from an ini file. FILE *fopen(const char *filename, const char *mode); as far as i know, fopen( ) accepts only...
25
by: subramanian100in | last post by:
Consider the following program: #include <stdio.h> #include <stdlib.h> int main(int argc, char *argv) { if (argc != 2) { printf("Usage: <program-name<text-file>\n");
3
by: mrmattborja | last post by:
Hello, Here is a program I'm playing around with for fun in the process of learning C. The objective is to create a function filesize() and call it from within the main() section to retrieve the...
6
by: chazzy69 | last post by:
So the function fopen() works most of the time heres the line im using- $file = fopen($nextPage, "r") or exit("Unable to open file!"); But for some reason that i haven't been able to...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.