Memory Question

[Clement]

Can i place variables or arrays in Disk instead of main memory........

[Chris Dollin]

Clement wrote:

Can i place variables or arrays in Disk instead of main memory........

Whether you can or can't, there's no way to specify it in standard C.

What problem are you trying to solve?

--
Is it a bird? It is a plane? No, it's: http://hpl.hp.com/conferences/juc2007/
"You've spotted a flaw in my thinking, Trev" Big Al,/The Beiderbeck Connection/

Hewlett-Packard Limited Cain Road, Bracknell, registered no:
registered office: Berks RG12 1HN 690597 England

[user923005]

On Mar 27, 7:40 am, "Clement" <[e-mail]:

Can i place variables or arrays in Disk instead of main memory........

You can write them to disk with fwrite() and read them with fread().
However, if you want to operate on them while they are on disk as
though they were in memory, then you will have to memory map them.
To find out how to memory map them (or even if it is possible to
memory map them) you will need to examine an operating system specific
newsgroup.

[Clement]

Chris Dollin wrote:

Clement wrote:

Can i place variables or arrays in Disk instead of main memory........

Whether you can or can't, there's no way to specify it in standard C.

What problem are you trying to solve?

--
Is it a bird? It is a plane? No, it's: http://hpl.hp.com/conferences/juc2007/
"You've spotted a flaw in my thinking, Trev" Big Al,/The Beiderbeck Connection/

Hewlett-Packard Limited Cain Road, Bracknell, registered no:
registered office: Berks RG12 1HN 690597 England

i want to create a binary tree[size of 1GB] so i want to place it in
Disk instead of memeory.. and i need to access through pointers.. can i

[Richard Heathfield]

Clement said:

<snip>

i want to create a binary tree[size of 1GB] so i want to place it in
Disk instead of memeory.. and i need to access through pointers.. can
i

Look up B+-trees in Knuth.

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at the above domain, - www.

[Chris Dollin]

Clement wrote:

Chris Dollin wrote:

>Clement wrote:

Can i place variables or arrays in Disk instead of main memory........

Whether you can or can't, there's no way to specify it in standard C.

What problem are you trying to solve?

--
Is it a bird? It is a plane? No, it's: http://hpl.hp.com/conferences/juc2007/
"You've spotted a flaw in my thinking, Trev" Big Al,/The Beiderbeck Connection/

Hewlett-Packard Limited Cain Road, Bracknell, registered no:
registered office: Berks RG12 1HN 690597 England

Snip signatures when responding, especially ridiculously long ones.

i want to create a binary tree[size of 1GB] so i want to place it in
Disk instead of memeory.. and i need to access through pointers.. can i

You can't do both in standard C (with C pointers). You'll have to give up
at least one of your criteria.

[1GB of what? The-here desktop machine has (fx:grope) 2Gb of memory; a one-
off 1Gb binary tree I could just read into a C program. You could just buy
more memory. (But remember who I work for ...)]

--
The second Jena user conference! http://hpl.hp.com/conferences/juc2007/
"Life is full of mysteries. Consider this one of them." Sinclair, /Babylon 5/

Hewlett-Packard Limited registered office: Cain Road, Bracknell,
registered no: 690597 England Berks RG12 1HN

[Ben Pfaff]

Chris Dollin <ch**********@hp.comwrites:

[1GB of what? The-here desktop machine has (fx:grope) 2Gb of memory; a one-
off 1Gb binary tree I could just read into a C program. You could just buy
more memory. (But remember who I work for ...)]

One of the places I work, I could file an Engineering Support
Services request online and in a few minutes a nice man would
show up with a handful of DIMMs and install them for me, no
questions asked. I imagine I could ask for 4 GB or so of RAM
before I'd need my manager to sign off on it.

But it's not normal, most of the time, to design software so that
it only runs efficiently with the latest and greatest and
highest-specced hardware. It's better, when you can, to make
your software work on the widest range of hardware that doesn't
require major algorithmic inconvenience.
--
Ben Pfaff
http://benpfaff.org

[Chris Dollin]

Ben Pfaff wrote:

Chris Dollin <ch**********@hp.comwrites:

>[1GB of what? The-here desktop machine has (fx:grope) 2Gb of memory; a one-
off 1Gb binary tree I could just read into a C program. You could just buy
more memory. (But remember who I work for ...)]

One of the places I work, I could file an Engineering Support
Services request online and in a few minutes a nice man would
show up with a handful of DIMMs and install them for me, no
questions asked. I imagine I could ask for 4 GB or so of RAM
before I'd need my manager to sign off on it.

But it's not normal, most of the time, to design software so that
it only runs efficiently with the latest and greatest and
highest-specced hardware. It's better, when you can, to make
your software work on the widest range of hardware that doesn't
require major algorithmic inconvenience.

I wasn't saying that the OP /should/ just go for more memory; just
pointing out that they're likely within the space of "do it all in
memory", if that's an available option. We don't know enough about
their context to know if it's realistic or not.

--
Yes, Virginia, there is a second Jena user conference: Palo Alto, Sep 2007.
"Anything can happen in the next half-hour." /Stingray/

Hewlett-Packard Limited registered office: Cain Road, Bracknell,
registered no: 690597 England Berks RG12 1HN

[Richard Tobin]

In article <11**********************@n59g2000hsh.googlegroups .com>,
Clement <je*******@gmail.comwrote:

>i want to create a binary tree[size of 1GB] so i want to place it in
Disk instead of memeory.. and i need to access through pointers.. can i

You could look to see if your system has functions for memory-mapping
a file (mmap() for example). But you will probably not be able to
guarantee that it is mapped at the same address in different runs of
the program, so it may well be more practical to use offsets rather
than pointers.

-- Richard
--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.

[SM Ryan]

"Clement" <je*******@gmail.comwrote:

# i want to create a binary tree[size of 1GB] so i want to place it in
# Disk instead of memeory.. and i need to access through pointers.. can i

Use a memory mapped file. You can use pointer operations on the file
contents. Whether a file page is actually on disk or in memory becomes
the operating system concerns.

Alternatives are pure memory where you malloc nodes. You need to write
tree traversal functions to translate external representations and
in memory version; or to do your own paging by maintianing page frames
and using seeks, reads, and writes to shuffle active nodes in and out
of memory.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
A bunch of savages in this town.

[Kenneth Brody]

SM Ryan wrote:

>
"Clement" <je*******@gmail.comwrote:

# i want to create a binary tree[size of 1GB] so i want to place it in
# Disk instead of memeory.. and i need to access through pointers.. can i

Use a memory mapped file. You can use pointer operations on the file
contents. Whether a file page is actually on disk or in memory becomes
the operating system concerns.

Note that memory-mapped files are beyond the scope of Standard C.
They are also system-specific, and not all systems support them.
If you do go that route, I would suggest at least putting wrappers
around the system-specific functions to allow easier porting to
other systems which support memory-mapped files differently.

[...]

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h|
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:Th*************@gmail.com>

[user923005]

On Mar 28, 1:38 pm, Kenneth Brody <[e-mail]:

SM Ryan wrote:

"Clement" <[e-mail]:

# i want to create a binary tree[size of 1GB] so i want to place it in
# Disk instead of memeory.. and i need to access through pointers.. can i

Use a memory mapped file. You can use pointer operations on the file
contents. Whether a file page is actually on disk or in memory becomes
the operating system concerns.

Note that memory-mapped files are beyond the scope of Standard C.
They are also system-specific, and not all systems support them.
If you do go that route, I would suggest at least putting wrappers
around the system-specific functions to allow easier porting to
other systems which support memory-mapped files differently.

Or find a toolkit that already does this.
I use ACE, but that is C++.
I seem to remember seeing similar things for C.

[Ed Prochak]

On Mar 28, 1:30 pm, SM Ryan <wyrm...@tango-sierra-oscar-foxtrot-
tango.fake.orgwrote:

"Clement" <[e-mail]:

# i want to create a binary tree[size of 1GB] so i want to place it in
# Disk instead of memeory.. and i need to access through pointers.. can i

Use a memory mapped file. You can use pointer operations on the file
contents. Whether a file page is actually on disk or in memory becomes
the operating system concerns.

Alternatives are pure memory where you malloc nodes. You need to write
tree traversal functions to translate external representations and
in memory version; or to do your own paging by maintianing page frames
and using seeks, reads, and writes to shuffle active nodes in and out
of memory.

OR use an operating system that does page swapping for you. Modern
ones all do, so unless you are runinng under some dumb custom kernal
OS, you should be able to just let the OD do the swapping to disc.

Ed