By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
460,028 Members | 881 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 460,028 IT Pros & Developers. It's quick & easy.

Accessing web with a C program

P: n/a
Neo
Hi,

I want to write a program that essentially connects to web,
access some web page and pass some parameters and fetch the return
values and print
like LWP in Perl Context.

Can some one suggest me what are the headers I might require and
directions to such a documentation.

Thanks,
Vasundhar

Feb 8 '07 #1
Share this Question
Share on Google+
37 Replies


P: n/a
Neo wrote:
Hi,

I want to write a program that essentially connects to web,
access some web page and pass some parameters and fetch the return
values and print
like LWP in Perl Context.

Can some one suggest me what are the headers I might require and
directions to such a documentation.
The functionality you require isn't part of standard C, but is part of
your operating environment. The best place to ask this question is on a
group dedicated to your platform of choice.

--
Ian Collins.
Feb 8 '07 #2

P: n/a
On Feb 8, 3:22 pm, "Neo" <bvasund...@gmail.comwrote:
Hi,

I want to write a program that essentially connects to web,
access some web page and pass some parameters and fetch the return
values and print
like LWP in Perl Context.

Can some one suggest me what are the headers I might require and
directions to such a documentation.
Given the proper compliant implementation, you should only need
#include <stdio.h>

You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.

However, few (if any) compliant C implementations provide this sort of
interpretation of the filename string. If you find that yours does
not, then you'll have to look for a compiler that does. Otherwise, you
are stuck with non-standard (as far as CLC goes) libraries that you
will have to discuss in the appropriate newsgroup (but not here).

HTH
--
Lew

Feb 8 '07 #3

P: n/a
Lew Pitcher <lp******@sympatico.cawrote:
You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
Should? I'm curious, what argues for this behavior? I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.

--
C. Benson Manica | I *should* know what I'm talking about - if I
cbmanica(at)gmail.com | don't, I need to know. Flames welcome.
Feb 8 '07 #4

P: n/a
Lew Pitcher wrote:
On Feb 8, 3:22 pm, "Neo" <bvasund...@gmail.comwrote:
>Hi,

I want to write a program that essentially connects to web,
access some web page and pass some parameters and fetch the return
values and print
like LWP in Perl Context.

Can some one suggest me what are the headers I might require and
directions to such a documentation.

Given the proper compliant implementation, you should only need
#include <stdio.h>

You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
Wow. stdio.h has really been gussied up since the last time I looked.
Feb 8 '07 #5

P: n/a
Neo wrote:
I want to write a program that essentially connects to web,
access some web page and pass some parameters and fetch the return
values and print
like LWP in Perl Context.

Can some one suggest me what are the headers I might require and
directions to such a documentation.
You need to look at the various HTTP protocol standards, among others.
You can probably find the source for curl and wget to give you pointers.
Google knows about all this.

Why are you reinventing this particular wheel?
Feb 8 '07 #6

P: n/a
On Thu, 8 Feb 2007 20:40:54 +0000 (UTC), in comp.lang.c , Christopher
Benson-Manica <at***@otaku.freeshell.orgwrote:
>Lew Pitcher <lp******@sympatico.cawrote:
>You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.

Should? I'm curious, what argues for this behavior?
One argument would be that since quite a long time ago, filesystems
have been distributed over different hardware.

In that context, http:// is no less a valid identifier of some
location on another machine than DRA1: or /mnt/hda1 or \\someserver or
whatever weirdness one would do on MVS to reference a disk on a remote
server.
I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.
Its an interesting point though. Why /should/ we expect fopen() to
handle a UNC or a mount point but not a URL?
--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan
Feb 8 '07 #7

P: n/a
Christopher Benson-Manica wrote, On 08/02/07 20:40:
Lew Pitcher <lp******@sympatico.cawrote:
>You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.

Should? I'm curious, what argues for this behavior?
The notation Lew has used is standard notation for specifying a web page
and is supported by a number of tools people use for browsing files on
their local file systems, therefore it would seem a logical way of
adding support for network access of files to standard C programs with
no need to change those programs. If could also support ftp:// smb:// etc.
I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.
I would not expect it either, but as Lew said short of the
implementation providing that functionality the OP will have to go to
system specific libraries.
--
Flash Gordon
Feb 8 '07 #8

P: n/a
Clever Monkey <cl**************@hotmail.com.invalidwrites:
Lew Pitcher wrote:
[...]
>You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
Wow. stdio.h has really been gussied up since the last time I looked.
Supporting URLs as file names shouldn't require any changes to
stdio.h. It would require substantial changes to the implementation
of fopen(), of course (unless the fopen() implementation in question
already provides this).

Whether stdio *should* support this is another question.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Feb 8 '07 #9

P: n/a
Mark McIntyre wrote:
>
.... snip ...
>
Its an interesting point though. Why /should/ we expect fopen()
to handle a UNC or a mount point but not a URL?
We don't. All we know is (from N869):

7.19.5.3 The fopen function

Synopsis

[#1]
#include <stdio.h>
FILE *fopen(const char * filename,
const char * mode);

Description

[#2] The fopen function opens the file whose name is the
string pointed to by filename, and associates a stream with
it.

Which says nothing whatsoever about the format of 'filename'. That
is system dependent.

--
<http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfocus.com/columnists/423>

"A man who is right every time is not likely to do very much."
-- Francis Crick, co-discover of DNA
"There is nothing more amazing than stupidity in action."
-- Thomas Matthews

Feb 8 '07 #10

P: n/a
In article <ln************@nuthaus.mib.org>,
Keith Thompson <ks***@mib.orgwrote:
>>You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
>Wow. stdio.h has really been gussied up since the last time I looked.
>Supporting URLs as file names shouldn't require any changes to
stdio.h. It would require substantial changes to the implementation
of fopen(), of course (unless the fopen() implementation in question
already provides this).
A practical implementation would probably choose to add quite a lot of
additional functionality. For example, it would probably want to
provide a way to determine the mime type and base URI of a resource
that had been opened in this way. Many protocols require
authentication of some kind - in the case of writing, almost always.
There are innumerable options related to caching and so on.

But I think it might well be possible to do it all in such a way that
a useful class of programs could indeed open files for reading just by
using fopen().

-- Richard
--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.
Feb 8 '07 #11

P: n/a
On Feb 8, 3:40 pm, Christopher Benson-Manica
<a...@otaku.freeshell.orgwrote:
Lew Pitcher <lpitc...@sympatico.cawrote:
You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.

Should? I'm curious, what argues for this behavior? I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.
What sort of behaviour would you expect from
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
or
FILE *somefile = fopen("/a/b/c.d","r");
?

What's different (wrt how fopen() is defined in the standard, or how
it is used in real life) betwen these usages and the usage I suggest?

What is your objection to
FILE *web = fopen("http://google.com/","r");
and why don't you voice similar objections to
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
?

Feb 9 '07 #12

P: n/a
On Feb 9, 1:22 am, "Neo" <bvasund...@gmail.comwrote:
Hi,

I want to write a program that essentially connects to web,
access some web page and pass some parameters and fetch the return
values and print
like LWP in Perl Context.
you may want to use,
1. libwww or curl or simillar stuff.(Google curl or libwww) which is
doing simillar stuff.
2. invoke external program from your c program
3. Use Browser API (if they are providing)

I think option 1 is good.
Ignore above option if you feel i dont understand your question
properly.
>
Can some one suggest me what are the headers I might require and
directions to such a documentation.
Third party libraries/headers
>
Thanks,
Vasundhar
--raxit

Feb 9 '07 #13

P: n/a
"Lew Pitcher" <lp******@sympatico.cawrote:
On Feb 8, 3:40 pm, Christopher Benson-Manica
<a...@otaku.freeshell.orgwrote:
Lew Pitcher <lpitc...@sympatico.cawrote:
You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
Should? I'm curious, what argues for this behavior? I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.

What sort of behaviour would you expect from
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
or
FILE *somefile = fopen("/a/b/c.d","r");
I'd expect it to open a local file.

This is very different from opening a URL, because that involves making
a connection to a random machine on the 'net, and possibly getting the
user to enter dialing passwords and all.
What's different (wrt how fopen() is defined in the standard, or how
it is used in real life) betwen these usages and the usage I suggest?
What's different between

cat /home/bloggsj/agenda

and

cat http://www.google.com/

entered on a normal Unixoid command line?
What is your objection to
FILE *web = fopen("http://google.com/","r");
I have no objection to it /per se/; but I do think that claiming that
you _should_ be able to do it is short-sighted. It would be a nice
enhancement in those environments where it makes sense; but it's hardly
a life-saver, and there are enough snags to it that it can hardly be
expected of most systems.

Richard
Feb 9 '07 #14

P: n/a
Mark McIntyre wrote:
>
On Thu, 8 Feb 2007 20:40:54 +0000 (UTC), in comp.lang.c , Christopher
Benson-Manica <at***@otaku.freeshell.orgwrote:
Lew Pitcher <lp******@sympatico.cawrote:
You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
Should? I'm curious, what argues for this behavior?

One argument would be that since quite a long time ago, filesystems
have been distributed over different hardware.

In that context, http:// is no less a valid identifier of some
location on another machine than DRA1: or /mnt/hda1 or \\someserver or
whatever weirdness one would do on MVS to reference a disk on a remote
server.
In that case, it's an O/S issue, and not a C issue.
I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.

Its an interesting point though. Why /should/ we expect fopen() to
handle a UNC or a mount point but not a URL?
Again, ask the O/S designers why they support UNC but not URL.

On the platforms that I use for which I have C runtime library source,
the C runtime library passes the "filename" as-is to the O/S. If a
UNC filename works, it is because the O/S itself will handle the UNC
parts, just as it is the O/S that handles any directory tree parts.

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h|
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:Th*************@gmail.com>
Feb 9 '07 #15

P: n/a

Mark McIntyre wrote:
On Thu, 8 Feb 2007 20:40:54 +0000 (UTC), in comp.lang.c , Christopher
Benson-Manica <at***@otaku.freeshell.orgwrote:
Lew Pitcher <lp******@sympatico.cawrote:
You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
Should? I'm curious, what argues for this behavior?

One argument would be that since quite a long time ago, filesystems
have been distributed over different hardware.

In that context, http:// is no less a valid identifier of some
location on another machine than DRA1: or /mnt/hda1 or \\someserver or
whatever weirdness one would do on MVS to reference a disk on a remote
server.
I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.

Its an interesting point though. Why /should/ we expect fopen() to
handle a UNC or a mount point but not a URL?
Maybe because the origins of fopen() predate the web by a decade. The
designer/s of fopen() probably couldn't forsee so that in a decade
"the network would be the computer."

Feb 9 '07 #16

P: n/a
On Thu, 08 Feb 2007 18:15:50 -0500, in comp.lang.c , CBFalconer
<cb********@yahoo.comwrote:
>Mark McIntyre wrote:
>>
... snip ...
>>
Its an interesting point though. Why /should/ we expect fopen()
to handle a UNC or a mount point but not a URL?

We don't. All we know is (from N869):
(snipped)
>
Which says nothing whatsoever about the format of 'filename'. That
is system dependent.
Exactly my point. There's nothing in the C standard that prohibits, or
encourages, fopen from handling URLs any differently from other
filenames.
--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan
Feb 9 '07 #17

P: n/a
On Fri, 09 Feb 2007 11:34:17 GMT, in comp.lang.c ,
rl*@hoekstra-uitgeverij.nl (Richard Bos) wrote:
>"Lew Pitcher" <lp******@sympatico.cawrote:
>What sort of behaviour would you expect from
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
or
FILE *somefile = fopen("/a/b/c.d","r");

I'd expect it to open a local file.
Then you don't know as much about unix as you thought you did.

On my machine, /apps/shared is a mountpoint on a netapp filer
somewhere about a hundred miles from me. I can also access this from
my Windows box as f:\shared.
>This is very different from opening a URL, because that involves making
a connection to a random machine on the 'net, and possibly getting the
user to enter dialing passwords and all.
As does the opening of a file on a networked drive, or through a
mount. There is no difference.

--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan
Feb 9 '07 #18

P: n/a
On Fri, 09 Feb 2007 13:06:45 -0500, in comp.lang.c , Kenneth Brody
<ke******@spamcop.netwrote:
>
In that case, it's an O/S issue, and not a C issue.
Strictly, its an implementation issue. An implementation of the C
library would be allowed to do this.
>>
Its an interesting point though. Why /should/ we expect fopen() to
handle a UNC or a mount point but not a URL?

Again, ask the O/S designers why they support UNC but not URL.
Again, ask the implementors, too.
>On the platforms that I use for which I have C runtime library source,
the C runtime library passes the "filename" as-is to the O/S. If a
UNC filename works, it is because the O/S itself will handle the UNC
parts, just as it is the O/S that handles any directory tree parts.
*shrug*. It happens that your OSen handle UNCs. If they didn't, your
library implementor could still support them.
--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan
Feb 9 '07 #19

P: n/a
rl*@hoekstra-uitgeverij.nl (Richard Bos) writes:
"Lew Pitcher" <lp******@sympatico.cawrote:
>On Feb 8, 3:40 pm, Christopher Benson-Manica
<a...@otaku.freeshell.orgwrote:
Lew Pitcher <lpitc...@sympatico.cawrote:
You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.

Should? I'm curious, what argues for this behavior? I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.

What sort of behaviour would you expect from
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
or
FILE *somefile = fopen("/a/b/c.d","r");

I'd expect it to open a local file.
How do you know that "/a/b/c.d" is the name of a local file? It could
well be stored on some server somewhere and accessed via NFS or some
other protocol.
This is very different from opening a URL, because that involves making
a connection to a random machine on the 'net, and possibly getting the
user to enter dialing passwords and all.
It's not obvious that fopen() should support the NFS protocol but not
the HTTP protocol. There is a difference in that an HTTP name (a URL)
specifally names the remote server -- but some systems, including VMS,
have file names that do the same thing but are expected to be treated
as ordinary files.

It's also possible, on some operating systems, to create a filesystem
that maps URLs to what appear to be local file names. On such a
system, fopen("http://google.com", "r") would invoke code in the
filesystem driver that would access google.com using the HTTP
protocol, but that would be invisible to the program that calls
fopen(). Given this capability letting fopen() accept URLs directly
is just a matter of doing the net access at a different level.

Of course, you have to draw the line somewhere. The usual practice is
*not* to treat URLs as file names. In my opinion, that's not a bad
place to draw the line, but I don't see any strong arguments for or
against drawing it somewhere else.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Feb 9 '07 #20

P: n/a
Lew Pitcher <lp******@sympatico.cawrote:
What is your objection to
FILE *web = fopen("http://google.com/","r");
and why don't you voice similar objections to
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
?
I suppose you have a point, although I still believe that the number
of edge cases (opening for "w" or "a" or "a+" etc., and what happens
if the file isn't found - the file system might mask the 404 error,
and the programmer might desire to parse the served 404 error rather
than get NULL from fopen()) argues against adding something as fraught
with additional complexity as HTTP requests to C.

--
C. Benson Manica | I *should* know what I'm talking about - if I
cbmanica(at)gmail.com | don't, I need to know. Flames welcome.
Feb 13 '07 #21

P: n/a
Mark McIntyre <ma**********@spamcop.netwrote:
On Fri, 09 Feb 2007 11:34:17 GMT, in comp.lang.c ,
rl*@hoekstra-uitgeverij.nl (Richard Bos) wrote:
"Lew Pitcher" <lp******@sympatico.cawrote:
What sort of behaviour would you expect from
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
or
FILE *somefile = fopen("/a/b/c.d","r");
I'd expect it to open a local file.

Then you don't know as much about unix as you thought you did.
That is undoubtedly true. However...
On my machine, /apps/shared is a mountpoint on a netapp filer
somewhere about a hundred miles from me. I can also access this from
my Windows box as f:\shared.
....on all systems I've used, Unix, M$ Windows, and otherwise, a network
drive was either available or not. If I'd logged in to the network from
before, it was there; if not, it was absent. This _may_ be true for
internet access, but it may also be different - think of dial-up
accounts, for example.

Note also the behaviour of the usual OS tools. You don't expect to pipe
a URL into your command line; you use wget for that. I can only assume
that this is for a reason. In any case, it's a matter of the OS, not of
the C library.

Richard
Feb 15 '07 #22

P: n/a
Keith Thompson <ks***@mib.orgwrote:
rl*@hoekstra-uitgeverij.nl (Richard Bos) writes:
This is very different from opening a URL, because that involves making
a connection to a random machine on the 'net, and possibly getting the
user to enter dialing passwords and all.

It's not obvious that fopen() should support the NFS protocol but not
the HTTP protocol.
Again: it is not fopen() which supports this in the first place. fopen()
gives you what it gets from the OS. To suggest that the C library
_should_ bypass the OS and support HTTP (how!?) when the OS only gives
you NFS as normal files is, IMO, rather weird.
It's also possible, on some operating systems, to create a filesystem
that maps URLs to what appear to be local file names. On such a
system, fopen("http://google.com", "r") would invoke code in the
filesystem driver that would access google.com using the HTTP
protocol, but that would be invisible to the program that calls
fopen(). Given this capability letting fopen() accept URLs directly
is just a matter of doing the net access at a different level.
Exactly: _if_ the OS gives you that. It's not fopen() doing this.
fopen() is just a gateway between your program and what the OS provides.

Richard
Feb 15 '07 #23

P: n/a
Richard Bos wrote:
Exactly: _if_ the OS gives you that. It's not fopen() doing this.
fopen() is just a gateway between your program and what the OS provides.
Surely that's not a /requirement/?

An implementation could provide a `fopen` which did all sorts of
interesting things not done by an underlying OS:open call. It
doesn't /have/ to be a direct gateway.

[It's probably better that it /is/, because otherwise it may have
to make all sorts of one-size-fails-to-fit-all choices that any
particular application could make better. Then again, it might
just provide an API which allowed one to attach arbitrary code
to certain kinds of file path and a library of useful but
replaceable arbitraries.]

--
Chris "electric hedgehog" Dollin
"Who are you? What do you want?" /Babylon 5/

Feb 15 '07 #24

P: n/a
In article <45****************@news.xs4all.nl>,
Richard Bos <rl*@hoekstra-uitgeverij.nlwrote:
>Again: it is not fopen() which supports this in the first place. fopen()
gives you what it gets from the OS. To suggest that the C library
_should_ bypass the OS and support HTTP (how!?) when the OS only gives
you NFS as normal files is, IMO, rather weird.
It would be bizarre for the OS (in the sense of the unix kernel) to
provide support for arbitrary protocols such as HTTP, since there's no
need for them to be implemented at that level. It would not make much
sense for it to be part of C either. But it would be perfectly
reasonable for a library to provide it, and for it to use the fopen()
function to do it. Obviously it would need various support functions,
because these protocols have many options that can't reasonably be
accessed through standard C functions, but it would be very useful if
existing programs that fopen() a file for reading were to suddenly
start working with, say, HTTP URIs.

-- Richard

--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.
Feb 15 '07 #25

P: n/a
ri*****@cogsci.ed.ac.uk (Richard Tobin) writes:
In article <45****************@news.xs4all.nl>,
Richard Bos <rl*@hoekstra-uitgeverij.nlwrote:
>>Again: it is not fopen() which supports this in the first place. fopen()
gives you what it gets from the OS. To suggest that the C library
_should_ bypass the OS and support HTTP (how!?) when the OS only gives
you NFS as normal files is, IMO, rather weird.

It would be bizarre for the OS (in the sense of the unix kernel) to
provide support for arbitrary protocols such as HTTP, since there's no
need for them to be implemented at that level. It would not make much
sense for it to be part of C either. But it would be perfectly
reasonable for a library to provide it, and for it to use the fopen()
function to do it. Obviously it would need various support functions,
because these protocols have many options that can't reasonably be
accessed through standard C functions, but it would be very useful if
existing programs that fopen() a file for reading were to suddenly
start working with, say, HTTP URIs.
Probably the most reasonable way to do this, on most systems, is to
provide a filesystem that's actually implemented on top of an http
interface, mapping URLs to file names somehow (the mapping could be
trivial or not). Some systems already provide hooks to support this
kind of thing. The details of implementing this kind of thing are, of
course, off-topic, but no more so than implementing a filesystem that
provides access to a local hard drive. Once it's done, fopen() and
friends will Just Work.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Feb 15 '07 #26

P: n/a
Keith Thompson wrote:
>
.... snip ...
>
Probably the most reasonable way to do this, on most systems, is
to provide a filesystem that's actually implemented on top of an
http interface, mapping URLs to file names somehow (the mapping
could be trivial or not). Some systems already provide hooks to
support this kind of thing. The details of implementing this
kind of thing are, of course, off-topic, but no more so than
implementing a filesystem that provides access to a local hard
drive. Once it's done, fopen() and friends will Just Work.
My PascalP runtime system of 25 years ago (which was also intended
to run C, but I never got around to the C compiler etc.)
implemented this through 7 arrays. The first tied device names to
device numbers. Anything beyond that table was considered to be a
disk file, and routed to the disk file system (if present). The
other 6 tables provided fopen, fclose, fread, fwrite, fstatus, and
fcntrl routine pointers.

With this I could easily tie any device into the system. One was a
LAN, which transmitted (and verified) medical test data to a
central system. Others included xy mapped terminals, terminals
with predetermined entry fields, printers with peculiar
requirements, A/D converters, etc. Everything was controlled by
the ISO standard procedures of Pascal.

Note the sly method of making this topical.

--
<http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfocus.com/columnists/423>

"A man who is right every time is not likely to do very much."
-- Francis Crick, co-discover of DNA
"There is nothing more amazing than stupidity in action."
-- Thomas Matthews
Feb 16 '07 #27

P: n/a
Chris Dollin <ch**********@hp.comwrote:
Richard Bos wrote:
Exactly: _if_ the OS gives you that. It's not fopen() doing this.
fopen() is just a gateway between your program and what the OS provides.

Surely that's not a /requirement/?
No. However, it's the usual case; and it's the only case the Standard
can assume. The original post in this sub-thread claimed that "You
/should/ be able to" read URLs using fopen(). My problem is not with the
possibility, but with the "should". Such a requirement would put a cruel
and unusual burden on the implementor, where the OS does not provide an
easy way to do it. If that implementor is mad enough to provide it
anyway, he's free to do it; but it can not and should not be expected as
a standard feature.

Richard
Feb 16 '07 #28

P: n/a
Richard Bos wrote:
Chris Dollin <ch**********@hp.comwrote:
>Richard Bos wrote:
Exactly: _if_ the OS gives you that. It's not fopen() doing this.
fopen() is just a gateway between your program and what the OS provides.

Surely that's not a /requirement/?

No. However, it's the usual case; and it's the only case the Standard
can assume.
The standard (do we mean the /users/ of the standard?) can't even assume
that. Of course a `fopen` that couldn't even open local files would be
pretty useless, or a `fopen` that deleted every vowel from the path [1];
but then someone else could similarly describe a http-deficient fopen.

S'all QoI.
The original post in this sub-thread claimed that "You
/should/ be able to" read URLs using fopen(). My problem is not with the
possibility, but with the "should". Such a requirement would put a cruel
and unusual burden on the implementor, where the OS does not provide an
easy way to do it. If that implementor is mad enough to provide it
anyway, he's free to do it; but it can not and should not be expected as
a standard feature.
I think you're reading the /should/ more strongly than you should;
I don't thing the earlier poster was speaking standardese.

[1] Although one that deleted every /space/ I could live with.

--
Chris "electric hedgehog" Dollin
"How am I to understand if you won't teach me?" - Trippa, /Falling/

Feb 16 '07 #29

P: n/a
Richard Tobin wrote:
In article <45****************@news.xs4all.nl>,
Richard Bos <rl*@hoekstra-uitgeverij.nlwrote:
>>Again: it is not fopen() which supports this in the first place. fopen()
gives you what it gets from the OS. To suggest that the C library
_should_ bypass the OS and support HTTP (how!?) when the OS only gives
you NFS as normal files is, IMO, rather weird.

It would be bizarre for the OS (in the sense of the unix kernel) to
provide support for arbitrary protocols such as HTTP, since there's no
need for them to be implemented at that level. It would not make much
sense for it to be part of C either. But it would be perfectly
reasonable for a library to provide it, and for it to use the fopen()
function to do it. Obviously it would need various support functions,
because these protocols have many options that can't reasonably be
accessed through standard C functions, but it would be very useful if
existing programs that fopen() a file for reading were to suddenly
start working with, say, HTTP URIs.

-- Richard
If you think about it - and we were all using URIs diligently, it wouldn't be
that bizarre of an idea.

Environments like KDE do this at a higher level allowing you to specify any
kind of URI as an argument to a file open, save, etc.

However, being that fopen only takes two arguments, it'd be really difficult
to make fly with the OS handling the underlying details. A complex protocol
like http has so many options (just look at libcurl) that you'd have to pass
a structure as the second argument.

The actual mapping of binary data received over a network connection specified
somehow via a URI of sort wouldn't be the difficult part.
Feb 16 '07 #30

P: n/a
Christopher Layne wrote:
However, being that fopen only takes two arguments, it'd be really difficult
to make fly with the OS handling the underlying details. A complex protocol
like http has so many options (just look at libcurl) that you'd have to pass
a structure as the second argument.

The actual mapping of binary data received over a network connection
specified somehow via a URI of sort wouldn't be the difficult part.
OT:
An example of how php does it:

http://us3.php.net/fopen

Also, this is one area where C++ really really shines.
Feb 16 '07 #31

P: n/a
In article <11************@news-west.n>,
Christopher Layne <cl****@com.anodizedwrote:
>However, being that fopen only takes two arguments, it'd be really difficult
to make fly with the OS handling the underlying details. A complex protocol
like http has so many options (just look at libcurl) that you'd have to pass
a structure as the second argument.
I agree that you would need additional interfaces, but I think that
for simply reading from an HTTP URI it would be sufficient to use
defaults so that fopen("http://example.com/foo", "r") would be useful.

Are there some particular options that you think need to be specified
often enough that this would not work?

-- Richard
--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.
Feb 16 '07 #32

P: n/a
Richard Tobin wrote:
I agree that you would need additional interfaces, but I think that
for simply reading from an HTTP URI it would be sufficient to use
defaults so that fopen("http://example.com/foo", "r") would be useful.

Are there some particular options that you think need to be specified
often enough that this would not work?

-- Richard
That's what I was thinking as well. That would be fine for a straight port 80
read. However, no one is going to settle for just that - and rightfully so,
they shouldn't.

1. HTTP POST.
2. SSL/HTTPS (read: nightmare)
3. HTTP 1.0 vs 1.1
4. Entire set of network timers affecting connect/read/write timeouts.

There would need to be a fsetopt() of some sort to make things work.
Feb 16 '07 #33

P: n/a
Christopher Layne wrote, On 16/02/07 14:35:
Richard Tobin wrote:
>I agree that you would need additional interfaces, but I think that
for simply reading from an HTTP URI it would be sufficient to use
defaults so that fopen("http://example.com/foo", "r") would be useful.

Are there some particular options that you think need to be specified
often enough that this would not work?

-- Richard

That's what I was thinking as well. That would be fine for a straight port 80
read. However, no one is going to settle for just that - and rightfully so,
they shouldn't.

1. HTTP POST.
Open in write or update mode with what you write being posted and what
you read being the result.
2. SSL/HTTPS (read: nightmare)
Open with "https://whatever" as the file name. That is all I have to do
with libcurl to make it use https instead of http.
3. HTTP 1.0 vs 1.1
Use 1.1 and have a C extension to switch to 1.0 if you really need it.
Note that a number of web sites (including http://clc-wiki.net/ ) will
not work with http 1.0 due to the methods used to host multiple web
sites on the same machine with only one IP address.
4. Entire set of network timers affecting connect/read/write timeouts.
Some kind of vaguely sensible default (which is what libcurl does) with
a C extension to override it.
There would need to be a fsetopt() of some sort to make things work.
You would "need" such an extension for two of the points you raised and
one of those is unlikely to be needed these days. OK, maybe 3 if you
need to do more for the POST.
--
Flash Gordon
Feb 17 '07 #34

P: n/a
Neo
<a...@otaku.freeshell.orgwrote:
Lew Pitcher <lpitc...@sympatico.cawrote:
You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
Should? I'm curious, what argues for this behavior? I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.

What sort of behaviour would you expect from
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
or
FILE *somefile = fopen("/a/b/c.d","r");
?

What's different (wrt how fopen() is defined in the standard, or how
it is used in real life) betwen these usages and the usage I suggest?

What is your objection to
FILE *web = fopen("http://google.com/","r");
and why don't you voice similar objections to
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
?
Vasundhar Quotes
if ((fp = fopen ("http://www.debian.com", "a+")) == NULL)
{
fprintf (stdout, "Can't open file.\n");
exit (1);
}
to make the life of you and many people contributed here with their
sweat and blood and fingers,
I personally coded and checked.
And the result was

curl-help/check_fopen
----------------------------------------------
Can't open file.

----------------------------------------------
Program exited successfully with errcode (1)
Press the Enter key to close this terminal ...

Thanks for the suggestions and so much learning I had during the
discussion.
Vasundhar.


Mar 13 '07 #35

P: n/a
Neo wrote:
<a...@otaku.freeshell.orgwrote:
Lew Pitcher <lpitc...@sympatico.cawrote:
You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
Should? I'm curious, what argues for this behavior? I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.
What's different (wrt how fopen() is defined in the standard, or how
it is used in real life) betwen these usages and the usage I suggest?

What is your objection to
FILE *web = fopen("http://google.com/","r");
and why don't you voice similar objections to
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
?

Vasundhar Quotes
if ((fp = fopen ("http://www.debian.com", "a+")) == NULL)
{
fprintf (stdout, "Can't open file.\n");
exit (1);
}
to make the life of you and many people contributed here with their
sweat and blood and fingers,
I personally coded and checked.
And the result was

curl-help/check_fopen
----------------------------------------------
Can't open file.

----------------------------------------------
Program exited successfully with errcode (1)
Press the Enter key to close this terminal ...
He, (Lew Pitcher), was talking about the possibility of the Standard
library fopen implementing access to remote files. From your code
snippet above, it looks like you've used routines from CURL, which is
not a part of the C Standard library. That was not the discussion of
this thread. Usage of third party libraries is mostly off-topic to
this group.

Mar 14 '07 #36

P: n/a
Neo
On Mar 14, 1:14 pm, "santosh" <santosh....@gmail.comwrote:
Neo wrote:
<a...@otaku.freeshell.orgwrote:
Lew Pitcher <lpitc...@sympatico.cawrote:
You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
Should? I'm curious, what argues for this behavior? I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.
What's different (wrt how fopen() is defined in the standard, or how
it is used in real life) betwen these usages and the usage I suggest?
What is your objection to
FILE *web = fopen("http://google.com/","r");
and why don't you voice similar objections to
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
?
Vasundhar Quotes
if ((fp = fopen ("http://www.debian.com", "a+")) == NULL)
{
fprintf (stdout, "Can't open file.\n");
exit (1);
}
to make the life of you and many people contributed here with their
sweat and blood and fingers,
I personally coded and checked.
And the result was
curl-help/check_fopen
----------------------------------------------
Can't open file.
----------------------------------------------
Program exited successfully with errcode (1)
Press the Enter key to close this terminal ...

He, (Lew Pitcher), was talking about the possibility of the Standard
library fopen implementing access to remote files. From your code
snippet above, it looks like you've used routines from CURL, which is
not a part of the C Standard library. That was not the discussion of
this thread. Usage of third party libraries is mostly off-topic to
this group.
Curl is not C.
above snippet is not curl.

Mar 21 '07 #37

P: n/a
Neo
On Mar 14, 1:14 pm, "santosh" <santosh....@gmail.comwrote:
Neo wrote:
<a...@otaku.freeshell.orgwrote:
Lew Pitcher <lpitc...@sympatico.cawrote:
You /should/ be able to
FILE *web = fopen("http://google.com/","r");
and have a valid read connection to the named web page.
Should? I'm curious, what argues for this behavior? I certainly
wouldn't expect it from fopen() regardless of whether it was commonly
implemented or not.
What's different (wrt how fopen() is defined in the standard, or how
it is used in real life) betwen these usages and the usage I suggest?
What is your objection to
FILE *web = fopen("http://google.com/","r");
and why don't you voice similar objections to
FILE *somefile = fopen("C:\\ABC\\DEF\\GHI.J","r");
or
FILE *somefile = fopen("DDNAME:SYSIN,"r");
?
Vasundhar Quotes
if ((fp = fopen ("http://www.debian.com", "a+")) == NULL)
{
fprintf (stdout, "Can't open file.\n");
exit (1);
}
to make the life of you and many people contributed here with their
sweat and blood and fingers,
I personally coded and checked.
And the result was
curl-help/check_fopen
----------------------------------------------
Can't open file.
----------------------------------------------
Program exited successfully with errcode (1)
Press the Enter key to close this terminal ...

He, (Lew Pitcher), was talking about the possibility of the Standard
library fopen implementing access to remote files. From your code
snippet above, it looks like you've used routines from CURL, which is
not a part of the C Standard library. That was not the discussion of
this thread. Usage of third party libraries is mostly off-topic to
this group.
Curl is not C.
above snippet is not curl.

Mar 21 '07 #38

This discussion thread is closed

Replies have been disabled for this discussion.