How can I create a Ram Disk within C/C++ program?

[Tom]

I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

Thanks in advance for any help.

[jmcgill]

Tom wrote:

I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

The particular semantics of a "ram disk" depend entirely on how your
operating system implements filesystems and/or storage devices.

I could go into great detail of how ram disks are implemented in C, in
the linux driver, but it would be way off topic for comp.lang.c.

Do you actually want a RAM disk, or do you merely want a memory
structure and access methods that work like a RAM disk but are local to
your program? Maybe what you really want is an in-memory database.

Or maybe you really do want to write a RAM disk. In that case you'd
better decide what platform you're using and ask in a forum appropriate
to it.

Somewhere, I have example code that includes a Windows NT RAM Disk
driver which is implemented in C++ and inline ASM.

[Knemon]

Tom wrote:

I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

Any ram disk you create with the fictional language C/C++ will be
fictional as well. As the author of your short story, you can create
that ram disk and determine the free space any way you wish.

If you choose to use an actual programming language like C (or its
bastard brother C++), you will find that such implementation-specific
details are not part of C (or C++). You will need to use
implementation- or platform-specific functionality, for which the
correct newsgroups are concerned with your implementation or platform,
not with the languages C or C++.

[Simon Biber]

Tom wrote:

I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

#include <stdio.h>
#include <stdlib.h>

long long create_ramdisk(void)
{
FILE *fp;
int ch;
long long space = 0;
system("mkdir -p /mnt/ramdisk");
system("/sbin/mke2fs -q /dev/ram");
system("mount /dev/ram /mnt/ramdisk");
system("df /mnt/ramdisk tmp.txt");
fp = fopen("tmp.txt", "r");
if(!fp)
{
printf("Can't open file\n");
return 0;
}
while((ch = fgetc(fp)) != EOF && ch != '\n');
while((ch = fgetc(fp)) != EOF && ch != ' ');
while((ch = fgetc(fp)) != EOF && ch == ' ');
while((ch = fgetc(fp)) != EOF && ch != ' ');
while((ch = fgetc(fp)) != EOF && ch == ' ');
while((ch = fgetc(fp)) != EOF && ch != ' ');
while((ch = fgetc(fp)) != EOF && ch == ' ');
ungetc(ch, fp);
if(feof(fp))
{
printf("Unexpected read\n");
return 0;
}
fscanf(fp, "%lld", &space);
fclose(fp);
remove("tmp.txt");
return space;
}

void remove_ramdisk(void)
{
system("umount /mnt/ramdisk");
}

int main(void)
{
long long space = create_ramdisk();
printf("Ramdisk has %lld bytes free\n", space);
remove_ramdisk();
return 0;
}

[sbiber@eagle c]$ gcc -std=c99 -pedantic -Wall -W -O2 ramdisk.c
[sbiber@eagle c]$ sudo ./a.out
Ramdisk has 14904 bytes free

--
Simon.

--
Posted via a free Usenet account from http://www.teranews.com

[Tom]

On Mon, 28 Aug 2006 05:10:48 GMT, Tom <Th********@earthlink.net>
wrote:

>I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

Thanks in advance for any help.

Yikes! -- I just realized how specific and problematic something I use
to do back in 1986 has become! Anybody remember the $1986 AT&T model
6300 sold in the year 1986? With that dual floppy 8 k byte ram green
monochrome monitor system I used a ram disk to speed things up. Floppy
drive reads were very slow. From the comments received I have gleaned
at least a few things.

1) The route I was hoping to take may not be the best choice.

2) The task is VERY sensitive to the operating system.

3) Some folks in here absolutely hate usage of the ++ tag onto there
favorite language C. Additionally they have little patience with
anyone who has diminutive skills compared to themselves. I certainly
apologized, if needed, for using the ++ and I really am trying to
become less stupid.

Even though it would be ideal for there to be an established news
group of experts for my particular challenge ... I will again wish for
some additional advice here in this group. After all ... my program
(that I have worked years on) is about 96% C and 4% ++. Additionally,
the ++ groups seem to be even faster in slamming up barriers and
crying out that you are in the wrong place! But again, what I hope to
do is within a C program and not something done purely at the
operating system level. Perhaps it is too cumbersome to create the
actual ram disk in C? But that still leaves my need to determine the
free space within the C code and provide logic that
manages the ram drive appropriately.

================================================== ===================
Additional information for those kind enough to have read this far >>

1) - I have a very large sequential data file. Approximately 400 M
byte in size. As I process this data I need to dynamically optimize on
a moving window of smaller size (approximately 20 M byte window size).
Specifically, the large file contains 20+ years of S&P index data and
the window represents a smaller time frame of this same data. The
optimization program is iterative and is used to generate settings for
forward performance evaluation. The program and the method are thus
totally blind to the future data and this provides a realistic test of
the method's long term robustness.

The optimization method iterates through the moving window 100's of
times before declaring the multi-dimensional, non-linear topology
space adequately searched. I have been using file pointers to reset to
the beginning of the moving window for each iteration. This works but
I have no clue about how efficient the cache management system is
working. I have no information on how many times blocks of data are
being read from the hard drive. I do know the program is slow. It can
take 60 hours to process 7 years of data while moving the edge of the
window every 3 months.

In the past I tried using very large storage arrays. My system has
sufficient memory ... but once I set the array size above
approximately 600,000 the program will not compile. This is
troublesome because Microsoft's help reference indicates that arrays
can be of size_t which provides a limit of 0x7CFFFFFF (or
2,097,151,999 -- if I correctly converted).

Most likely there are better ways to solve the problem than using a
Ram disk, but I am clueless to what they may be. I am certainly open
to all suggestions!!

The program does work when digesting the full data file. I have tuned
up the entire 20+ years to one set of parameters successfully;
however, this in itself does not demonstrate dynamic tuning
efficiency.

I was hoping to put the moving window data into ram to speed up data
throughput and possible reduce hard drive wear and tear. When I run
the program; about 1/2 of my system's 1 Gig of memory is not being
used ... So there is plenty of memory available. The program is to
clear the memory space and reload data only when the window is moved.
Not when simply iterating through the data for the tuning of that
specific time period. Also the program is to include logic to prevent
writing too much data to the ram disk.

2) I am using Win 2000 and Visual C++.NET.

3) Whether you can provide helpful suggestions or not ... I thank you
having read the above. Best wishes to all.

-- Tom

[Tom]

On Mon, 28 Aug 2006 20:30:31 +1000, Simon Biber <te******@ralmin.cc>
wrote:

>Tom wrote:

>I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

#include <stdio.h>
#include <stdlib.h>

long long create_ramdisk(void)
{
FILE *fp;
int ch;
long long space = 0;
system("mkdir -p /mnt/ramdisk");
system("/sbin/mke2fs -q /dev/ram");
system("mount /dev/ram /mnt/ramdisk");
system("df /mnt/ramdisk tmp.txt");
fp = fopen("tmp.txt", "r");
if(!fp)
{
printf("Can't open file\n");
return 0;
}
while((ch = fgetc(fp)) != EOF && ch != '\n');
while((ch = fgetc(fp)) != EOF && ch != ' ');
while((ch = fgetc(fp)) != EOF && ch == ' ');
while((ch = fgetc(fp)) != EOF && ch != ' ');
while((ch = fgetc(fp)) != EOF && ch == ' ');
while((ch = fgetc(fp)) != EOF && ch != ' ');
while((ch = fgetc(fp)) != EOF && ch == ' ');
ungetc(ch, fp);
if(feof(fp))
{
printf("Unexpected read\n");
return 0;
}
fscanf(fp, "%lld", &space);
fclose(fp);
remove("tmp.txt");
return space;
}

void remove_ramdisk(void)
{
system("umount /mnt/ramdisk");
}

int main(void)
{
long long space = create_ramdisk();
printf("Ramdisk has %lld bytes free\n", space);
remove_ramdisk();
return 0;
}

[sbiber@eagle c]$ gcc -std=c99 -pedantic -Wall -W -O2 ramdisk.c
[sbiber@eagle c]$ sudo ./a.out
Ramdisk has 14904 bytes free

--
Simon.

================================================== ================

Holy smokes!!!

While I was writing a reply to the first two responses ... Simon
posted what appears to be a total solution!!

I am groggy from lack of sleep and must rest before I test it ... but
I absolutely must acknowledge Simon's help.

Thank you, Thank you, Thank you !!!
Truthfully, some of the syntax and instructions in Simon's example are
new to me. I am already learning just from having read through it!

Please, where did you get this code? If you are the author ... you da
man! :) If you have an online source you referenced for this example
.... oh please share that too!

Redundantly, but well deserved ... Thank you again.

[pete]

Tom wrote:

>
On Mon, 28 Aug 2006 20:30:31 +1000, Simon Biber <te******@ralmin.cc>
wrote:

Tom wrote:

I'd greatly appreciate advice and
code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

#include <stdio.h>
#include <stdlib.h>

long long create_ramdisk(void)
{
FILE *fp;
int ch;
long long space = 0;
system("mkdir -p /mnt/ramdisk");
system("/sbin/mke2fs -q /dev/ram");
system("mount /dev/ram /mnt/ramdisk");
system("df /mnt/ramdisk tmp.txt");
fp = fopen("tmp.txt", "r");
if(!fp)
{
printf("Can't open file\n");
return 0;
}
while((ch = fgetc(fp)) != EOF && ch != '\n');
while((ch = fgetc(fp)) != EOF && ch != ' ');
while((ch = fgetc(fp)) != EOF && ch == ' ');
while((ch = fgetc(fp)) != EOF && ch != ' ');
while((ch = fgetc(fp)) != EOF && ch == ' ');
while((ch = fgetc(fp)) != EOF && ch != ' ');
while((ch = fgetc(fp)) != EOF && ch == ' ');
ungetc(ch, fp);
if(feof(fp))
{
printf("Unexpected read\n");
return 0;
}
fscanf(fp, "%lld", &space);
fclose(fp);
remove("tmp.txt");
return space;
}

void remove_ramdisk(void)
{
system("umount /mnt/ramdisk");
}

int main(void)
{
long long space = create_ramdisk();
printf("Ramdisk has %lld bytes free\n", space);
remove_ramdisk();
return 0;
}

[sbiber@eagle c]$ gcc -std=c99 -pedantic -Wall -W -O2 ramdisk.c
[sbiber@eagle c]$ sudo ./a.out
Ramdisk has 14904 bytes free

--
Simon.

================================================== ================

Holy smokes!!!

While I was writing a reply to the first two responses ... Simon
posted what appears to be a total solution!!

I am groggy from lack of sleep and must rest before I test it ... but
I absolutely must acknowledge Simon's help.

Thank you, Thank you, Thank you !!!

Truthfully, some of the syntax and instructions in Simon's example are
new to me. I am already learning just from having read through it!

I thought that Simon was being sarcastic
and providing an example of noportable code
for the purpose of illustrating that this was
a system specific problem and not a C problem.

--
pete

[Simon Biber]

Tom wrote:

On Mon, 28 Aug 2006 20:30:31 +1000, Simon Biber <te******@ralmin.cc>
wrote:

[...]

> system("mkdir -p /mnt/ramdisk");
system("/sbin/mke2fs -q /dev/ram");
system("mount /dev/ram /mnt/ramdisk");
system("df /mnt/ramdisk tmp.txt");

While I was writing a reply to the first two responses ... Simon
posted what appears to be a total solution!!

If you had actually read and understood my code, you'd see that it is a
bit of a joke. It is pure C code, and does actually create a ramdisk,
but it only works on Linux. All the real work is done by Linux commands
that are called through the 'system' function.

Unfortunately I don't believe a similar solution is possible on Windows.
You need to actually write a driver or use someone's existing ramdisk
driver. That's rather difficult, and of course, very system specific.

I recommend you ask in a place that specifically discusses Windows
programming.

--
Simon.

--
Posted via a free Usenet account from http://www.teranews.com

[Ico]

Tom <Th********@earthlink.netwrote:

On Mon, 28 Aug 2006 20:30:31 +1000, Simon Biber <te******@ralmin.cc>
wrote:

>>Tom wrote:

>>I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

[ snip code ]

Holy smokes!!!

>
While I was writing a reply to the first two responses ... Simon
posted what appears to be a total solution!!

I am groggy from lack of sleep and must rest before I test it ... but
I absolutely must acknowledge Simon's help.

Thank you, Thank you, Thank you !!!

I guess somebody here is oblivious of sarcasm.
--
:wq
^X^Cy^K^X^C^C^C^C

[xenonysf]

ok guys;

i prefer u to look at memory mapped i/o api functions
for such things at win32...

Ico wrote:

Tom <Th********@earthlink.netwrote:

On Mon, 28 Aug 2006 20:30:31 +1000, Simon Biber <te******@ralmin.cc>
wrote:

>Tom wrote:
I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

[ snip code ]

Holy smokes!!!

While I was writing a reply to the first two responses ... Simon
posted what appears to be a total solution!!

I am groggy from lack of sleep and must rest before I test it ... but
I absolutely must acknowledge Simon's help.

Thank you, Thank you, Thank you !!!

I guess somebody here is oblivious of sarcasm.
--
:wq
^X^Cy^K^X^C^C^C^C

[Kenny McCormack]

In article <44***********@mindspring.com>,
pete <pf*****@mindspring.comwrote:
....

>Truthfully, some of the syntax and instructions in Simon's example are
new to me. I am already learning just from having read through it!

I thought that Simon was being sarcastic

Gee. ya think?

>and providing an example of noportable code

I think the point is that Simon's code was entirely portable.
And, like most entirely portable code, utterly useless.

>for the purpose of illustrating that this was
a system specific problem and not a C problem.

[Kenny McCormack]

In article <44**********************@free.teranews.com>,
Simon Biber <te******@ralmin.ccwrote:

>Tom wrote:

>On Mon, 28 Aug 2006 20:30:31 +1000, Simon Biber <te******@ralmin.cc>
wrote:

[...]

>> system("mkdir -p /mnt/ramdisk");
system("/sbin/mke2fs -q /dev/ram");
system("mount /dev/ram /mnt/ramdisk");
system("df /mnt/ramdisk tmp.txt");

While I was writing a reply to the first two responses ... Simon
posted what appears to be a total solution!!

If you had actually read and understood my code, you'd see that it is a
bit of a joke.

The thing about sarcasm is that it only works on a defenseless opponent.
Or, TPIAW, it fizzles when it is met in kind.

>Unfortunately I don't believe a similar solution is possible on Windows.
You need to actually write a driver or use someone's existing ramdisk
driver. That's rather difficult, and of course, very system specific.

Actually, there's no difference. The thing you are missing is that the
RAM disk driver is built-in in (more or less recent versions of) Linux.
So, all you are doing is invoking the user-land functions that control
the kernel driver (*).

Just as it is in DOS/Windows as long as the appropriated DEVICE=
line exists in CONFIG.SYS (or whatever the current equivalent is).

(*) Note that there exist similar drivers in DOS/Windows - where a
user-land utility controls the kernel driver.

[Kenny McCormack]

In article <44***********************@dreader32.news.xs4all.n l>,
Ico <us****@zevv.nlwrote:

>Tom <Th********@earthlink.netwrote:

>On Mon, 28 Aug 2006 20:30:31 +1000, Simon Biber <te******@ralmin.cc>
wrote:

>>>Tom wrote:
I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

[ snip code ]

Holy smokes!!!

>>
While I was writing a reply to the first two responses ... Simon
posted what appears to be a total solution!!

I am groggy from lack of sleep and must rest before I test it ... but
I absolutely must acknowledge Simon's help.

Thank you, Thank you, Thank you !!!

I guess somebody here is oblivious of sarcasm.

Yes. You.

[Flash Gordon]

Tom wrote:

On Mon, 28 Aug 2006 05:10:48 GMT, Tom <Th********@earthlink.net>
wrote:

>I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

Thanks in advance for any help.

Yikes! -- I just realized how specific and problematic something I use
to do back in 1986 has become! Anybody remember the $1986 AT&T model
6300 sold in the year 1986? With that dual floppy 8 k byte ram green
monochrome monitor system I used a ram disk to speed things up. Floppy
drive reads were very slow.

It's no more difficult and no less system specific than it was back
then. The difference is that if you are doing it to speed things up you
are quite possibly wasting your time and effort. Ever head of caching?
It's something some OSs are quite good at.

From the comments received I have gleaned
at least a few things.

1) The route I was hoping to take may not be the best choice.

True. If the user wants your data files on a RAM disk they can create
one and put them there. Such as on Linux making /tmp a ram disk.

2) The task is VERY sensitive to the operating system.

Yes.

3) Some folks in here absolutely hate usage of the ++ tag onto there
favorite language C.

Not necessarily. It is just that C++ is a very different language with
its own new group. So C++ is no more or less topical here than Fortran,
ML, TMS320C25 assembler or any of the other languages other than C that
I have used.

Additionally they have little patience with
anyone who has diminutive skills compared to themselves. I certainly

No, people have a lot of patience with those who have little knowledge
or skill. We have recently had some threads where people with little
knowledge or skill but who wanted to learn and were asking topical (if
simple) questions where the posts were not merely answered but the
original posters complimented. The difference is they were asking C
questions.

apologized, if needed, for using the ++ and I really am trying to
become less stupid.

Even though it would be ideal for there to be an established news
group of experts for my particular challenge ... I will again wish for
some additional advice here in this group.

Complaining at us is not the best method I've ever seen of getting advice.

After all ... my program
(that I have worked years on) is about 96% C and 4% ++. Additionally,
the ++ groups seem to be even faster in slamming up barriers and
crying out that you are in the wrong place! But again, what I hope to
do is within a C program and not something done purely at the
operating system level.

C knows nothing of disks, file systems, space, caching, ram disks or
lots of other stuff. It does know about streams, but which the file name
means is entirely up to the system and nothing to do with C.

Perhaps it is too cumbersome to create the
actual ram disk in C?

You can't in C and you never have been able to. You may be able to do it
with OS specific extensions but that discussion belongs in an OS
specific group.

But that still leaves my need to determine the
free space within the C code and provide logic that
manages the ram drive appropriately.

You can't with standard C. You will have to ask how to do that in a
group dedicated to your system.

<snip>

working. I have no information on how many times blocks of data are
being read from the hard drive. I do know the program is slow. It can
take 60 hours to process 7 years of data while moving the edge of the
window every 3 months.

Then profile it. Until you have *measured* and *know* what the bottle
neck is there is no point in doing anything.

In the past I tried using very large storage arrays. My system has
sufficient memory ... but once I set the array size above
approximately 600,000 the program will not compile. This is
troublesome because Microsoft's help reference indicates that arrays
can be of size_t which provides a limit of 0x7CFFFFFF (or
2,097,151,999 -- if I correctly converted).

Since size_t is not a number I'm and SIZE_MAX is unlikely to be
0x7CFFFFFF I suspect you have misread it. Ask in a Windows news group
since all we know is it is allowing arrays a lot larger than the
standard says it has to.

Most likely there are better ways to solve the problem than using a
Ram disk, but I am clueless to what they may be. I am certainly open
to all suggestions!!

<snip>

Then I *seriously* suggest you profile it. It is possible that even
Windows can manage caching well enough these days and that your problem
is something different.
--
Flash Gordon
Being helpful despite the attitude shown at the start of Tom's post not
because of it.

[Ico]

Kenny McCormack <ga*****@xmission.xmission.comwrote:

In article <44***********************@dreader32.news.xs4all.n l>,
Ico <us****@zevv.nlwrote:

>>Tom <Th********@earthlink.netwrote:

>>On Mon, 28 Aug 2006 20:30:31 +1000, Simon Biber <te******@ralmin.cc>
wrote:

Tom wrote:
I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.
>
I also need to be able to determine the free space.

[ snip code ]

Holy smokes!!!

>>>
While I was writing a reply to the first two responses ... Simon
posted what appears to be a total solution!!

I am groggy from lack of sleep and must rest before I test it ... but
I absolutely must acknowledge Simon's help.

Thank you, Thank you, Thank you !!!

I guess somebody here is oblivious of sarcasm.

Yes. You.

Re-reading the previous posts I must admit you are right. I guess for me
being a nerd with few social skills it is easier to recognize sarcasm in
code then in normal language. Scary.

--
:wq
^X^Cy^K^X^C^C^C^C

[Simon Biber]

Kenny McCormack wrote:

In article <44***********@mindspring.com>,
pete <pf*****@mindspring.comwrote:
...

>>Truthfully, some of the syntax and instructions in Simon's example are
new to me. I am already learning just from having read through it!

I thought that Simon was being sarcastic

Gee. ya think?

Call it sarcasm, or a joke, or perhaps just taking things too literally.

I'm still not sure whether Tom's reply was being sarcastic in return or
if he was really thanking me without realising that the code was useless
to him.

>and providing an example of noportable code

I think the point is that Simon's code was entirely portable.
And, like most entirely portable code, utterly useless.

Well, if you're running it on a Linux system, it's not utterly useless.
It does achieve what it set out to do. The error handling could be
greatly improved, and there are certainly better ways to get the free
space on a file system than trying to parse the output of 'df'.

By the way: my program reported a value as bytes that is actually the
number of kilobytes free. If you want bytes, add a --block-size=1 option
to the df call.

--
Simon.

[jmcgill]

Simon Biber wrote:

Tom wrote:

>I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

system("mkdir -p /mnt/ramdisk");
system("/sbin/mke2fs -q /dev/ram");
system("mount /dev/ram /mnt/ramdisk");
system("df /mnt/ramdisk tmp.txt");

I'm going to feel like a total asshole if this was actually what the OP
had in mind.

[jmcgill]

Flash Gordon wrote:

It's no more difficult and no less system specific than it was back
then. The difference is that if you are doing it to speed things up you
are quite possibly wasting your time and effort. Ever head of caching?
It's something some OSs are quite good at.

It still makes sense on diskless clients that boot over NFS and so on.
It's also indicated in some applications where security is sensitive, or
for devices that need to operate in environments where moving parts are
impractical.

[Default User]

xenonysf wrote:

ok guys;

Please don't top-post. Your reply belongs following or interspersed
with properly trimmed quotes.

Brian

[Flash Gordon]

jmcgill wrote:

Flash Gordon wrote:

>It's no more difficult and no less system specific than it was back
then. The difference is that if you are doing it to speed things up you
are quite possibly wasting your time and effort. Ever head of caching?
It's something some OSs are quite good at.

It still makes sense on diskless clients that boot over NFS and so on.
It's also indicated in some applications where security is sensitive, or
for devices that need to operate in environments where moving parts are
impractical.

I agree there are good reasons for it sometimes. I just don't believe
that they necessarily apply to the OP. The maxim of measure first, on
the other hand, definitely does apply.
--
Flash Gordon

[Kenny McCormack]

In article <44**********@news.peopletelecom.com.au>,
Simon Biber <ne**@ralmin.ccwrote:
....

>Well, if you're running it on a Linux system, it's not utterly useless.

But in the religion of this group, if it isn't 120% guaranteed to work
on not only every machine/platform in existence, but also every single
machine/platform conceivable by the human mind, then it ain't worth
spitting on... (but that doesn't stop them from doing so)

[Frederick Gotham]

Tom posted:

I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

Thanks in advance for any help.

struct RamDisk { int i; }

int main(void)
{
struct RamDisk obj;
return 0;
}

--

Frederick Gotham

[Tom]

On Mon, 28 Aug 2006 17:31:11 GMT, Simon Biber <ne**@ralmin.ccwrote:

>Kenny McCormack wrote:

>In article <44***********@mindspring.com>,
pete <pf*****@mindspring.comwrote:
...

>>>Truthfully, some of the syntax and instructions in Simon's example are
new to me. I am already learning just from having read through it!
I thought that Simon was being sarcastic

Gee. ya think?

Call it sarcasm, or a joke, or perhaps just taking things too literally.

I'm still not sure whether Tom's reply was being sarcastic in return or
if he was really thanking me without realising that the code was useless
to him.

I "Truly" *was* and still am thanking you. In my opinion sarcasm is an
inefficient way of teaching people. Certainly among experts sarcasm at
the level suggested could catch you just right and be gut busting
funny; however, expert I will never be. I think you were being helpful
and provided a perfect solution for the environment which you use.
Certainly illustrates a powerful feature within Linux.

In my ignorance I did not initially define which operating system or C
compiler I was using. Your reply appears to me as magical because I
had never seen the "system" commands used before. I have never used
Linux. Without the additional comments stating how non-portable your
solution was ... I would have been trying to get it to work on my Win
2000 set-up. To think that my knowledge is so limited that I would
have actually attempted this ... really is funny!

Other bits that further illuminate my ignorance are >>

The usage of:

long long space = create_ramdisk();

I am guessing you are "naming" or creating a "space" of some sort
within the program. I have read in the past about "named" spaces but
that topic really went over my head at that time and remains there
today. Until now ... I have been able to work around not understanding
named spaces but I should revisit that subject soon. Perhaps now it
will sink in.

Another bit that caught my eye was the "%lld" usage. I have used "%ld"
many times but have never even seen double precision specified in such
manner.

At this time I need to pursue whatever it takes to make this Ram disk
feature work in Win 2000. One Win 2000 specific article I have found
is the following:

http://support.microsoft.com/kb/q257405/

This task might become hairball nasty difficult. I really don't know.
Any suggestions on good books that bridge between C code and beginner
level system programming?

Comparing the solution for a Win 2000 system to that of a Linux
solution will be most interesting and educational for me. Using a Ram
disk for security reasons (Kiosk) or harsh environments (high
vibration) is very interesting. Even if it is not the best solution
for my specific task I am learning a little outside the box. :)

Some have suggested profiling to determine the exact bottlenecks. I
have no profiling experience but understand a slow I/O task made fast
using a Ram disk or other technique may not speed up the program any
what-so-ever if it was not previously the rate limiting step ... and
if the I/O task is truly performed parallel to the other tasks.
However, if the I/O has any sequential behavior ... even if it is not
the longest duration event ... saving time on that specific
sequentially acting task speeds the program. It seems to me that a Ram
disk would make read time as fast as possible and thus making read
time no longer a consideration for speed improvements. However, would
not cache management still be handling the I/O from the Ram disk? So
another method of some sort might be a better solution? A method that
simply retrieves values designated as in memory and entirely avoids
any internal cache management logic? After all, why should cache
memory be used to hold something already residing in memory? Even if
the time penalty is insignificant ... it is still double handling the
data.

My original attempts at specifying a very large array for this task
failed. Anytime I exceeded approximately 600,000 for the array size
.... the program would not compile.

Quoting from MSDN Library >>
----------------------------
Largest Array Size
ANSI 3.3.3.4, 4.1.1 The type of integer required to hold the maximum
size of an array—that is, the size of size_t

The size_t typedef is an unsigned int with the range 0x00000000 to
0x7CFFFFFF.
----------------------------

The above range yeilds a maximum indexing value of 2,097,151,999. Am I
misreading the doco somehow?

Some have suggested some type of memory mapping technique. I have done
some searching in that area and have not yet found a technical reason
for memory mapping to be of benefit. Its name certainly sounds
promissing ... so I will continue searching in that area too.

Boost provides some library routines too. I have not used any Boost
routines as of yet; however, the boost.filesystem provides some
interesting functionality ... but still does not manage the memory
space as I am hoping to do.

>>and providing an example of noportable code

I think the point is that Simon's code was entirely portable.
And, like most entirely portable code, utterly useless.

I'm confused how any code that is "entirely" portable can be
considered to be useless?

>Well, if you're running it on a Linux system, it's not utterly useless.
It does achieve what it set out to do. The error handling could be
greatly improved, and there are certainly better ways to get the free
space on a file system than trying to parse the output of 'df'.

By the way: my program reported a value as bytes that is actually the
number of kilobytes free. If you want bytes, add a --block-size=1 option
to the df call.

[Tom]

On Mon, 28 Aug 2006 23:35:25 GMT, Frederick Gotham
<fg*******@SPAM.comwrote:

>Tom posted:

>I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

Thanks in advance for any help.

struct RamDisk { int i; }

int main(void)
{
struct RamDisk obj;
return 0;
}

Fred -- I fail to see how a structure that shares the name of the task
I need to perform and has a single integer for a member can manage a
relatively large block of data from a huge data file.

If you want to populate that structure with a few doubles and ints ...
then try it as an array of size 1,000,000 or so ...

struct RamDisk obj[1000000];

.... then you'd be right back where my compiler has already choked.

[Chris Torek]

>Flash Gordon wrote:

>>[RAM-disk creation and manipulation is] no more difficult and no
less system specific than it was back then.

Well, no less system-specific, to be sure. Sometimes "more (or
less) difficult" depending on the system. (Aside: in vxWorks, just
include the "ram disk" component in your project. The default name
is "/ram0" but you can change it. If you want more than one, it is
somewhat more difficult.)

>>The difference is that if you are doing it to speed things up you
are quite possibly wasting your time and effort.

Indeed.

In article <H5GIg.4036$y61.1915@fed1read05>
jmcgill <jm*****@email.arizona.eduwrote:

>It still makes sense on diskless clients that boot over NFS and so on.
It's also indicated in some applications where security is sensitive, or
for devices that need to operate in environments where moving parts are
impractical.

Sometimes, a physical (as opposed to virtual, in-kernel-only) RAM
disk is appropriate. That is, you have a piece of hardware you
plug in, which looks and acts like a disk as far as the system is
concerned, but actually stores data in RAM.

While many systems (including vxWorks, as mentioned above) do
include in-main-memory pseudo-disk devices, in a purely theoretical
sense, there is *never* [%] any reason to use one: whatever you
are doing that is going through some sort of "file system" layer,
only to wind up going directly to regular old ordinary memory, is
going through a "unnecessary" manipulation. File systems do a lot
of work in order to deal with structured, and somewhat klunky, disk
drive hardware, where memory exists in the form of "disk blocks" or
"sectors" that can only be rewritten in entire units, cannot be
moved or resized, and is generally very slow to access (about 5 or
6 decimal orders of magnitude slower than RAM). Real RAM does not
share most of these characteristic drawbacks, and going through
a (usually quite heavy) software layer that simply pretends to add
some of them back is just silly.

[% One obvious exception is when the RAM-disk is being used for
testing the file system software. Here, even in theory, the RAM
drive is useful. :-) ]

In practice, the reason for using a RAM disk is that the stubborn
software (written by some stubborn and/or or long-gone programmer)
insists on using the file system interface, and the system provides
no way to short-circuit this. For instance, in C -- to get at
least marginally back on topic -- one might have a program (or
large library) that does all its I/O to a "stdio" stream. Since
ISO C has no way to "open memory as a stream", one is often forced
to supply an actual, on-disk file. To get decent performance, one
may wish to use an "on-RAM-disk file" instead of "on-actual-disk".

Note that it can make sense to use a "RAM file system" rather than
a "RAM disk" in this case. Going through a *real* file system,
you may do a whole lot of extra work trying to avoid talking too
much to a slow (real) disk -- i.e., spend large amounts of CPU time
to avoid waiting for the device. But if the "device" is fake, and
actually is very fast (relative to real disks), this CPU time is
being wasted. It may be better to go through a fake file system
(one which uses RAM), instead of a real file system on a fake disk.

Of course, it would be even better still if C had some sort of
memory-oriented stdio streams -- or something like the 4.4BSD
"funopen", which is even more general. (I am pretty sure that
funopen() did get proposed for C99; but we might consider ourselves
lucky to have gotten even snprintf(). :-) )
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) [phone]
email: forget about it http://web.torek.net/torek/index.html
Reading email is like searching for food in the garbage, thanks to spammers.

[Nick Keighley]

Tom wrote:

On Mon, 28 Aug 2006 17:31:11 GMT, Simon Biber <ne**@ralmin.ccwrote:

Kenny McCormack wrote:

<snip>

I think the point is that Simon's code was entirely portable.
And, like most entirely portable code, utterly useless.

I'm confused how any code that is "entirely" portable can be
considered to be useless?

Kenny McCormack makes silly remarks on comp.lang.c just to get a
reaction. This is known as "trolling"; it is best to ignore him.

<snip>
--
Nick Keighley

[bali]

jmcgill wrote:

Tom wrote:

I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

The particular semantics of a "ram disk" depend entirely on how your
operating system implements filesystems and/or storage devices.

I could go into great detail of how ram disks are implemented in C, in
the linux driver, but it would be way off topic for comp.lang.c.

Do you actually want a RAM disk, or do you merely want a memory
structure and access methods that work like a RAM disk but are local to
your program? Maybe what you really want is an in-memory database.

Or maybe you really do want to write a RAM disk. In that case you'd
better decide what platform you're using and ask in a forum appropriate
to it.

Somewhere, I have example code that includes a Windows NT RAM Disk
driver which is implemented in C++ and inline ASM.

[Frederick Gotham]

Tom posted:

>>struct RamDisk { int i; }

int main(void)
{
struct RamDisk obj;
return 0;
}

Fred -- I fail to see how a structure that shares the name of the task
I need to perform and has a single integer for a member can manage a
relatively large block of data from a huge data file.

As I. I also fail to see how ramdisks would be topical on comp.lang.c.

If you want to allocate a load of memory, then use "malloc". If you want to
give it a drive letter, then go to a newsgroup which discusses your platform.

--

Frederick Gotham

[Tom]

On 29 Aug 2006 07:39:06 GMT, Chris Torek <no****@torek.netwrote:

>>Flash Gordon wrote:

>>>[RAM-disk creation and manipulation is] no more difficult and no
less system specific than it was back then.

Well, no less system-specific, to be sure. Sometimes "more (or
less) difficult" depending on the system. (Aside: in vxWorks, just
include the "ram disk" component in your project. The default name
is "/ram0" but you can change it. If you want more than one, it is
somewhat more difficult.)

>>>The difference is that if you are doing it to speed things up you
are quite possibly wasting your time and effort.

Indeed.

In article <H5GIg.4036$y61.1915@fed1read05>
jmcgill <jm*****@email.arizona.eduwrote:

>>It still makes sense on diskless clients that boot over NFS and so on.
It's also indicated in some applications where security is sensitive, or
for devices that need to operate in environments where moving parts are
impractical.

Sometimes, a physical (as opposed to virtual, in-kernel-only) RAM
disk is appropriate. That is, you have a piece of hardware you
plug in, which looks and acts like a disk as far as the system is
concerned, but actually stores data in RAM.

While many systems (including vxWorks, as mentioned above) do
include in-main-memory pseudo-disk devices, in a purely theoretical
sense, there is *never* [%] any reason to use one: whatever you
are doing that is going through some sort of "file system" layer,
only to wind up going directly to regular old ordinary memory, is
going through a "unnecessary" manipulation. File systems do a lot
of work in order to deal with structured, and somewhat klunky, disk
drive hardware, where memory exists in the form of "disk blocks" or
"sectors" that can only be rewritten in entire units, cannot be
moved or resized, and is generally very slow to access (about 5 or
6 decimal orders of magnitude slower than RAM). Real RAM does not
share most of these characteristic drawbacks, and going through
a (usually quite heavy) software layer that simply pretends to add
some of them back is just silly.

[% One obvious exception is when the RAM-disk is being used for
testing the file system software. Here, even in theory, the RAM
drive is useful. :-) ]

In practice, the reason for using a RAM disk is that the stubborn
software (written by some stubborn and/or or long-gone programmer)
insists on using the file system interface, and the system provides
no way to short-circuit this. For instance, in C -- to get at
least marginally back on topic -- one might have a program (or
large library) that does all its I/O to a "stdio" stream. Since
ISO C has no way to "open memory as a stream", one is often forced
to supply an actual, on-disk file. To get decent performance, one
may wish to use an "on-RAM-disk file" instead of "on-actual-disk".

Note that it can make sense to use a "RAM file system" rather than
a "RAM disk" in this case. Going through a *real* file system,
you may do a whole lot of extra work trying to avoid talking too
much to a slow (real) disk -- i.e., spend large amounts of CPU time
to avoid waiting for the device. But if the "device" is fake, and
actually is very fast (relative to real disks), this CPU time is
being wasted. It may be better to go through a fake file system
(one which uses RAM), instead of a real file system on a fake disk.

Of course, it would be even better still if C had some sort of
memory-oriented stdio streams -- or something like the 4.4BSD
"funopen", which is even more general. (I am pretty sure that
funopen() did get proposed for C99; but we might consider ourselves
lucky to have gotten even snprintf(). :-) )

Well written and detailed. Thank you Chris. I agree fully with you and
wish to elaborated below a little to make sure I have not missed the
boat.

I have revisited creating a very large array for holding raw data
which is used in an iterative optimization algorithm. The purpose of
the large array is to eliminate the slow hard disk reads and increase
program throughput. On an older machine (Win2000, Visual C++ 6.0, 256
M ram) when I compile with a huge array I get the following warning:

================================================== ===================
warning LNK4084: total image size 343744512 exceeds max (268435456);
image may not run
================================================== ===================

The help doco on a newer machine (Win2000, Visual C++.NET, 1 G ram)
indicates that the image can be as large as 1 G byte. (This machine is
in the middle of a multi-day run and I do not have a compilation
error/warning available at this time for this system.)

To get around the image size limits and to speed up I/O it seems a ram
disk is one valid approach. A search of ram disk drivers shows that
some can create disks 64+ Gig in size.

I am guessing on my older machine if I were to increase the ram that I
would still be up against the same image size limitation. Thus the ram
disk provides a great improvement for I/O for an iterative type
process and allows you to work around image size limits?

I wish I knew how to load the data into a block of memory and have it
treated as if were the cache for the hard drive. In other words, ready
to use and no manipulation needed. No double handling: caching,
paging, etc.

Chris's above reference to wishing for "funopen" combined with my
image size problems leads me to believe that a ram disk is my only
valid option at this time? Unless I were skilled enough to write my
own funopen! That'll be the day!! :)

In the future the 64 bit machines may allow much larger image sizes?
Then the data can loaded into arrays as long as memory resources are
available. And only if the data is in nicely proportioned blocks (i.e.
a data structure.) Arrays would not be the optimal solution for loose
form, varying length or delimited type of data. In that case you
really need to be able to assign a block of ram for I/O. I found the
following 64 bit article interesting:

http://msdn.microsoft.com/vstudio/ja...4/default.aspx

Thus for my immediate needs and current inability to write my own
"funopen" ... I must use a ram disk to speed up the I/O on my data
hungry application?

Can anyone recommend their favorite vendor for a ram disk driver?
There seem to be several available but I have yet to find a review on
which is the best and most stable.

Thanks. -- Tom

[Kenny McCormack]

In article <11*********************@b28g2000cwb.googlegroups. com>,
Nick Keighley <ni******************@hotmail.comwrote:

>Tom wrote:

>On Mon, 28 Aug 2006 17:31:11 GMT, Simon Biber <ne**@ralmin.ccwrote:

>Kenny McCormack wrote:

<snip>

>I think the point is that Simon's code was entirely portable.
And, like most entirely portable code, utterly useless.

I'm confused how any code that is "entirely" portable can be
considered to be useless?

....
Nick Keighley makes stupid racist comments on comp.lang.c just to
prove what an idiot he is. This is known as "typical behavior for him";
it is best to make your own choices, as a free rational adult.

[Thomas J. Gritzan]

Tom schrieb:

I have revisited creating a very large array for holding raw data
which is used in an iterative optimization algorithm. The purpose of
the large array is to eliminate the slow hard disk reads and increase
program throughput. On an older machine (Win2000, Visual C++ 6.0, 256
M ram) when I compile with a huge array I get the following warning:

================================================== ===================
warning LNK4084: total image size 343744512 exceeds max (268435456);
image may not run
================================================== ===================

Thats a linker error. What did you do to the linker?

Just don't use a filescope array, try to malloc() a block of memory, or if
malloc() can't handle that large blocks, allocate multiple chunks of
smaller memory blocks.

You really don't need a ramdisk.

--
Thomas

[pete]

Nick Keighley wrote:

Kenny McCormack makes silly remarks on comp.lang.c just to get a
reaction. This is known as "trolling"; it is best to ignore him.

I agree.

--
pete

[Tom]

On Wed, 30 Aug 2006 00:18:26 +0200, "Thomas J. Gritzan"
<Ph*************@gmx.dewrote:

>Tom schrieb:

>I have revisited creating a very large array for holding raw data
which is used in an iterative optimization algorithm. The purpose of
the large array is to eliminate the slow hard disk reads and increase
program throughput. On an older machine (Win2000, Visual C++ 6.0, 256
M ram) when I compile with a huge array I get the following warning:

================================================= ====================
warning LNK4084: total image size 343744512 exceeds max (268435456);
image may not run
================================================= ====================

Thats a linker error. What did you do to the linker?

Just don't use a filescope array, try to malloc() a block of memory, or if
malloc() can't handle that large blocks, allocate multiple chunks of
smaller memory blocks.

You really don't need a ramdisk.

From Tom to Tom --

Thanks!

My linker is just in the default setting mode. I have never adjusted
it. I write "console" apps.

I need to wait until my current run finishes (another painful 28
hrs)... but as soon as it does I will trying to malloc() my way to
success.

If this works ... I will be thrilled!

[Malcolm]

"Tom" <Th********@earthlink.netwrote in message
news:bt********************************@4ax.com...

I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

Thanks in advance for any help.

To create ramdisk

unsigned char * ramdisk = malloc(1024 * 64);

if that proves too small

temp = realloc(ramdisk, 1024 * 128);
if(temp)
ramdisk = temp;
else
/* tell user you are very sorry but there is no more memory */

Now we can implement
int ramputc(int ch)
{
if( ch == EOF)
/* do you wnat to handle this ? */
if(len < capacity)
ramdisk[len++] = (unsigned char) ch;
else
return EOF;
return 0;
}

ramfputs, ramfwrite and so on can be built on top of ramputc.

What you cannot do is associate your ramdisk with a C FILE *. Very stupid
restriction, I know. In C++ you can theoretically override the IoStream
classes, but rather you than me.

--
www.personal.leeds.ac.uk/~bgy1mm
freeware games to download.

[osmium]

"Tom" writes:

I'd greatly appreciate advice and code snippets on how to create a ram
disk within a C/C++ program.

I also need to be able to determine the free space.

I haven't followed this thread, so this may just repeat what someone has
already said.

In order to create a Ram Disk on a modern system you need the ability to
"nail" some space in the RAM caching mechanism provided by the OS. Lacking
such a capability, the OS is free to roll the contents of what you think is
a RAM address out to disk.

The OS people have already done what they think is a decent job of handling
both internal and external memory via. RAM caching and disk caching.
Anything you do is going to have to fight the mechanisms already in place.
The ability to nail is, IMO, a requisite of an adult OS, which should allow
some real time usage on the system intermixed with the ordinary flow of
lower priority jobs. IOW, the ability might be offered on something like,
say, Windows NT but not on the ordinary line of OSes. I am only familiar
with one disk caching system, and it did not provide an ability to either
nail or tack - where a tack is the short term equivalent of a nail.