Any way to capture stdcout output to memory?

[Jim Langston]

I had asked this in comp.lang.c++ with out any answers that would actually
work, so I'm hoping someone here may know a way.

I am calling C library functions that want to output to stdout. I need to
capture this to memory for use internally inside the program without
modifying the library calls that actually do the output.

This is for the GraphViz libraries and it is the output of dot that I'm
actually trying to capture. I have compiled all the libraries and the dot
exectuable which outputs to stdcout. There is also an option to output to a
file and if someone knows a way of redirecting a FILE* to memory that would
work too. There is an option to output to memory but it is a static buffer
that is WAY too small and is broken anyway. When trying this way I get a
memory error and tracing through the code somewhere deep inside the library
it's attempting to write to a FILE*

If no one knows of a way I will have to write a libary function to do this
and have to delve into the GraphViz libraries and try not to come across the
same bug that the pervious person who was attempting to write to memory
does.

One thing I will do in my attempt is not allocate the memory inside the
library as is currently attempted, but pass in a function pointer to
allocate the memory (that will probably just wrap malloc and/or remalloc).

Any help is appreciated. The GraphViz library is pure C.

--
Jim Langston
ta*******@rocketmail.com

[Richard Heathfield]

Jim Langston said:

I had asked this in comp.lang.c++ with out any answers that would
actually
work, so I'm hoping someone here may know a way.

I am calling C library functions that want to output to stdout. I need
to capture this to memory for use internally inside the program without
modifying the library calls that actually do the output.

Surely this sounds like a job for a pipe? Not a standard C concept,
granted, but I'll bet you that comp.unix.programmer will know all about
it.

--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Jim Langston]

Richard Heathfield wrote:

Jim Langston said:

> I had asked this in comp.lang.c++ with out any answers that would
actually
work, so I'm hoping someone here may know a way.

I am calling C library functions that want to output to stdout. I
need to capture this to memory for use internally inside the program
without modifying the library calls that actually do the output.

Surely this sounds like a job for a pipe? Not a standard C concept,
granted, but I'll bet you that comp.unix.programmer will know all
about it.

I know about pipes from the OS point of view. The thing is I only one one
process running. My program will be calling the library functions a lot as
it produces it's own data to be worked on, then needs to work on the results
from the library calls that dot uses.

--
Jim Langston
ta*******@rocketmail.com

[Richard Heathfield]

Jim Langston said:

Richard Heathfield wrote:

>Jim Langston said:

<snip>

>>>
I am calling C library functions that want to output to stdout. I
need to capture this to memory for use internally inside the program
without modifying the library calls that actually do the output.

Surely this sounds like a job for a pipe? Not a standard C concept,
granted, but I'll bet you that comp.unix.programmer will know all
about it.

I know about pipes from the OS point of view. The thing is I only one

[want?]

one process running. My program will be calling the library functions
a lot
as it produces it's own data to be worked on, then needs to work on the
results from the library calls that dot uses.

Then I think you're going to have to resort to some lib-hackery. Sorry.

--
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Jim Langston]

Richard Heathfield wrote:

Jim Langston said:

>Richard Heathfield wrote:

>>Jim Langston said:

<snip>

>>>>
I am calling C library functions that want to output to stdout. I
need to capture this to memory for use internally inside the
program without modifying the library calls that actually do the
output.

Surely this sounds like a job for a pipe? Not a standard C concept,
granted, but I'll bet you that comp.unix.programmer will know all
about it.

I know about pipes from the OS point of view. The thing is I only
one

[want?]

>one process running. My program will be calling the library
functions a lot
as it produces it's own data to be worked on, then needs to work on
the results from the library calls that dot uses.

Then I think you're going to have to resort to some lib-hackery.
Sorry.

I was actually playing around with createpipe and such and trying to get the
code to work when I finally realized I was trying to redirect stdin to
stdout and stdout to stdin. Heh, that would be an endless loop since the
functoin reads from stdin.

I think I might have figured out a way, something like:

functiontocapturestdout();
functionthatwritestostdout();
functionthtaclosescapture();

functiontocapturestdout() would actually have to create a thread and have
stdout redirected to that thread. That thread would then read from the
redirected stdout and throw the data into a buffer.
functiontoclosecapture() would end the process.

This seems doable, but now seems OT for this newsgroup and I will start
discuessing the possibilty in microsoft.language.vc.language.

Thank you for leading me into the right direction.
--
Jim Langston
ta*******@rocketmail.com

[Willem]

Jim wrote:
) I know about pipes from the OS point of view. The thing is I only one one
) process running. My program will be calling the library functions a lot as
) it produces it's own data to be worked on, then needs to work on the results
) from the library calls that dot uses.

You can use a pipe from a single process, but only if that pipe buffers
enough data so that your library call won't block on the write to stdout.
If the lib only produces a line at a time that may be possible. Read
your platform's documentation on pipe() and dup2().

You might also conceivably be able to link the 'write' call in the library
to your own function, but that is very platform specific deep black magic.

Otherwise, your best bet is pointing stdout to a file and then reading
from it.
SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT

[Jim Langston]

Willem wrote:

Jim wrote:
) I know about pipes from the OS point of view. The thing is I only
one one ) process running. My program will be calling the library
functions a lot as ) it produces it's own data to be worked on, then
needs to work on the results ) from the library calls that dot uses.

You can use a pipe from a single process, but only if that pipe
buffers enough data so that your library call won't block on the
write to stdout.
If the lib only produces a line at a time that may be possible.

That's not going to be possible. On a small sample set of data 215 lines
were produced.

Read your platform's documentation on pipe() and dup2().

You might also conceivably be able to link the 'write' call in the
library
to your own function, but that is very platform specific deep black
magic.

Yeah, very deep black magic. Trying to get something not quite that deep,
but I will if I have too.

Otherwise, your best bet is pointing stdout to a file and then reading
from it.

Unfortunately, I need this to be rather fast and file IO being a major
bottleneck isn't going to work for me. I think I may be able to get away
with running the call itself it it's own process allowing me to redirect the
stdout. Not sure yet.

Thanks.

--
Jim Langston
ta*******@rocketmail.com

[Kenneth Brody]

Jim Langston wrote:

>
Richard Heathfield wrote:

Jim Langston said:

[...]

I am calling C library functions that want to output to stdout. I
need to capture this to memory for use internally inside the program
without modifying the library calls that actually do the output.

Surely this sounds like a job for a pipe? Not a standard C concept,
granted, but I'll bet you that comp.unix.programmer will know all
about it.

I know about pipes from the OS point of view. The thing is I only one one
process running. My program will be calling the library functions a lot as
it produces it's own data to be worked on, then needs to work on the results
from the library calls that dot uses.

Then you'll need to resort to some further OT-for-clc method, such
as connecting stdout to a pipe, and creating a thread to read the
other side of the pipe.

How to do this is very OT here, but may be topical for something like
comp.unix.programmer.

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h|
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:Th*************@gmail.com>

[Clever Monkey]

Jim Langston wrote:

Richard Heathfield wrote:

>Jim Langston said:

>> I had asked this in comp.lang.c++ with out any answers that would
actually
work, so I'm hoping someone here may know a way.

I am calling C library functions that want to output to stdout. I
need to capture this to memory for use internally inside the program
without modifying the library calls that actually do the output.

Surely this sounds like a job for a pipe? Not a standard C concept,
granted, but I'll bet you that comp.unix.programmer will know all
about it.

I know about pipes from the OS point of view. The thing is I only one one
process running. My program will be calling the library functions a lot as
it produces it's own data to be worked on, then needs to work on the results
from the library calls that dot uses.

Not standard C, but you can diddle the file descriptors using dup2(),
saving the old FD and switching it back when you are done.
--
clvrmnky <mailto:sp******@clevermonkey.org>

Direct replies to this address will be blacklisted. Replace "spamtrap"
with my name to contact me directly.

[CBFalconer]

Jim Langston wrote:

>
I had asked this in comp.lang.c++ with out any answers that would
actually work, so I'm hoping someone here may know a way.

I am calling C library functions that want to output to stdout.
I need to capture this to memory for use internally inside the
program without modifying the library calls that actually do the
output.

All the functions that output to stdout do so as if the final
output is implemented by "putc(i, stdout)". So all you have to do
is write replacement functions, such as, say, "dputs(char *)". It
will call both putc and your own putcm. putcm stuffs the char copy
in the memory buffer. Then modify your source to call the new
functions in place of the old.

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.
** Posted from http://www.teranews.com **

[Jim Langston]

CBFalconer wrote:

Jim Langston wrote:

>>
I had asked this in comp.lang.c++ with out any answers that would
actually work, so I'm hoping someone here may know a way.

I am calling C library functions that want to output to stdout.
I need to capture this to memory for use internally inside the
program without modifying the library calls that actually do the
output.

All the functions that output to stdout do so as if the final
output is implemented by "putc(i, stdout)". So all you have to do
is write replacement functions, such as, say, "dputs(char *)". It
will call both putc and your own putcm. putcm stuffs the char copy
in the memory buffer. Then modify your source to call the new
functions in place of the old.

".. modify your source..."

If I was able to modify the source I'd have it write to a memory buffer in
the first place. The problem is the actual output is being produced deep
inside a library call. This right now is not an option, although I may have
to make it an option if I can't find a good way to do it. I think one of
the methods of redirecting stdout will work, although it looks like I'll
have to use a thread which I don't like but *shrug*

--
Jim Langston
ta*******@rocketmail.com

[Kenny McCormack]

In article <ra***********@newsfe06.lga>,
Jim Langston <ta*******@rocketmail.comwrote:
....

>".. modify your source..."

If I was able to modify the source I'd have it write to a memory buffer in
the first place. The problem is the actual output is being produced deep
inside a library call. This right now is not an option, although I may have
to make it an option if I can't find a good way to do it. I think one of
the methods of redirecting stdout will work, although it looks like I'll
have to use a thread which I don't like but *shrug*

I, too, have hit this issue. First comment: You (Jim) have made it
clear that the platforms is Windows, not Unix, right? I think some of
the posters are assuming Unix. Unix would simplify things, since these
sorts of things are always easier there. Things like dup(), etc,
although often implemented in Windows, often don't work quite the same
under Windows as under Unix.

On Windows, I was able to solve the problem (very OT for clc, of course)
by diddling with the elements of the the FILE structure. The basic
trick is that you set the buffersize very high, so that it _never_ gets
written out. Then, your main program extracts the information out of
the stdout buffer (pointed to by one of the elements in the FILE
structure). I did this using the MinGW compiler - but I would imagine
it is doable with most compilers.

I think that the thread solution is actually better, but is, as you say,
complicated to implement. I wish somebody would do it, though, and make
the code publicly available.

P.S. Note that even if source is available and that modifying it to
call different functions was an option, it is possible that doing so
would be too difficult - if, for example, the existing code makes many,
many calls to stdout, intermixing different functions, etc.

[Richard Tobin]

In article <fv**********@news.xmission.com>,
Kenny McCormack <ga*****@xmission.xmission.comwrote:

>On Windows, I was able to solve the problem (very OT for clc, of course)
by diddling with the elements of the the FILE structure. The basic
trick is that you set the buffersize very high, so that it _never_ gets
written out.

If you're willing to do that, there's a much more elegant solution for
some stdio implementations (I don't know about Windows). These
implementations store pointers to the underlying i/o functions in the
FILE struct, so by modifying them you can arrange for your own
function to be called when the data is flushed. Typically there will
be some kind of cookie in which you could store, say, a buffer to
which the data would be copied.

For example, MacOS-X's FILE structure (which I think it inherits
from FreeBSD) includes:

/* operations */
void *_cookie; /* cookie passed to io functions */
int (*_close)(void *);
int (*_read) (void *, char *, int);
fpos_t (*_seek) (void *, fpos_t, int);
int (*_write)(void *, const char *, int);

I would be nice if this was standardised, since it's something that
people have been wanting to do for decades.

-- Richard
--
:wq

[Chris Torek]

In article <fv***********@pc-news.cogsci.ed.ac.uk>
Richard Tobin <ri*****@cogsci.ed.ac.ukwrote:

>... there's a much more elegant solution for
some stdio implementations (I don't know about Windows). These
implementations store pointers to the underlying i/o functions in the
FILE struct, so by modifying them you can arrange for your own
function to be called when the data is flushed.

But these are not exposed to users, for various reasons; you
are only supposed to know about funopen() (and several shorter
names for specific cases; funopen() is the general case).

>For example, MacOS-X's FILE structure (which I think it inherits
from FreeBSD) includes:

/* operations */
void *_cookie; /* cookie passed to io functions */
int (*_close)(void *);
int (*_read) (void *, char *, int);
fpos_t (*_seek) (void *, fpos_t, int);
int (*_write)(void *, const char *, int);

This came from my stdio by way of 4.4BSD and then (presumably)
FreeBSD, yes.

>I would be nice if this was standardised, since it's something that
people have been wanting to do for decades.

This lets you do:

struct whatever some_data_struct;
int read_op(void *, char *, int);
int write_op(void *, const char *, int);

... set up "some_data_struct" as needed ...
fp = funopen(&some_data_struct, read_op, write_op, NULL, NULL);
... handle failure ...
some_library_function(fp); /* uses fprintf() and/or fgets() */
... etc ...

so that you can capture the output of "well-behaved" library
functions that write to a stdio "FILE *", feed input to "well-behaved"
library functions that read from a "FILE *", and so on.

In the OP's case, however, he is calling a "poorly-behaved" library
function that uses a "global variable"[%], namely stdout. Since
stdout need not be a modifiable lvalue, you cannot simply do:

fflush(stdout);
saved_stdout = stdout;
stdout = fp;
bad_library_function();
fflush(stdout);
stdout = saved_stdout;

If you are willing to go for non-portable solutions, the sequence:

target_fd = open_or_pipe_or_whatever();
... check for failures ...
saved_stdout_fd = dup(STDOUT_FILENO);
... check for failures ...

fflush(stdout);
dup2(target_fd, STDOUT_FILENO);
bad_library_function();
fflush(stdout);
dup2(saved_stdout_fd, STDOUT_FILENO);

will do the trick on POSIX systems. The dup2() calls changes the
"global variable"[%] STDOUT_FILENO "behind the back" of the C
library stdio. The pre- and post-call fflush()es make sure that
all stdout output goes to the "correct" fd. The non-portable
version that reassigns the "global variable" stdout uses the same
technique: push out all the "correct" output, make a "global
variable" change, call the badly behaved function with the system's
global state appropriately rearranged, push out *that* output, then
rearrange the global state once more to put it back to normal.

[% The words "global variable" are in quotes because C does not
define the term, and in fact stdout and POSIX_FILENO may well be
constants, not variables. In practice, stdout is either an "address
constant", like &__sstdout, or an actual variable but possibly
const-qualfied, while POSIX_FILENO is simply the integer constant
1. Nonetheless, the POSIX stdout descriptor is a "global variable",
by my definition. It is changeable via system call: dup2(fd, 1)
changes it.]
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) [phone]
email: gmail (figure it out) http://web.torek.net/torek/index.html