473,322 Members | 1,352 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Memory Question

Can i place variables or arrays in Disk instead of main memory........

Mar 27 '07 #1
12 1590
Clement wrote:
Can i place variables or arrays in Disk instead of main memory........
Whether you can or can't, there's no way to specify it in standard C.

What problem are you trying to solve?

--
Is it a bird? It is a plane? No, it's: http://hpl.hp.com/conferences/juc2007/
"You've spotted a flaw in my thinking, Trev" Big Al,/The Beiderbeck Connection/

Hewlett-Packard Limited Cain Road, Bracknell, registered no:
registered office: Berks RG12 1HN 690597 England

Mar 27 '07 #2
On Mar 27, 7:40 am, "Clement" <jeba.r...@gmail.comwrote:
Can i place variables or arrays in Disk instead of main memory........
You can write them to disk with fwrite() and read them with fread().
However, if you want to operate on them while they are on disk as
though they were in memory, then you will have to memory map them.
To find out how to memory map them (or even if it is possible to
memory map them) you will need to examine an operating system specific
newsgroup.

Mar 28 '07 #3

Chris Dollin wrote:
Clement wrote:
Can i place variables or arrays in Disk instead of main memory........

Whether you can or can't, there's no way to specify it in standard C.

What problem are you trying to solve?

--
Is it a bird? It is a plane? No, it's: http://hpl.hp.com/conferences/juc2007/
"You've spotted a flaw in my thinking, Trev" Big Al,/The Beiderbeck Connection/

Hewlett-Packard Limited Cain Road, Bracknell, registered no:
registered office: Berks RG12 1HN 690597 England
i want to create a binary tree[size of 1GB] so i want to place it in
Disk instead of memeory.. and i need to access through pointers.. can i

Mar 28 '07 #4
Clement said:

<snip>
i want to create a binary tree[size of 1GB] so i want to place it in
Disk instead of memeory.. and i need to access through pointers.. can
i
Look up B+-trees in Knuth.

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at the above domain, - www.
Mar 28 '07 #5
Clement wrote:
Chris Dollin wrote:
>Clement wrote:
Can i place variables or arrays in Disk instead of main memory........

Whether you can or can't, there's no way to specify it in standard C.

What problem are you trying to solve?

--
Is it a bird? It is a plane? No, it's: http://hpl.hp.com/conferences/juc2007/
"You've spotted a flaw in my thinking, Trev" Big Al,/The Beiderbeck Connection/

Hewlett-Packard Limited Cain Road, Bracknell, registered no:
registered office: Berks RG12 1HN 690597 England
Snip signatures when responding, especially ridiculously long ones.
i want to create a binary tree[size of 1GB] so i want to place it in
Disk instead of memeory.. and i need to access through pointers.. can i
You can't do both in standard C (with C pointers). You'll have to give up
at least one of your criteria.

[1GB of what? The-here desktop machine has (fx:grope) 2Gb of memory; a one-
off 1Gb binary tree I could just read into a C program. You could just buy
more memory. (But remember who I work for ...)]

--
The second Jena user conference! http://hpl.hp.com/conferences/juc2007/
"Life is full of mysteries. Consider this one of them." Sinclair, /Babylon 5/

Hewlett-Packard Limited registered office: Cain Road, Bracknell,
registered no: 690597 England Berks RG12 1HN

Mar 28 '07 #6
Chris Dollin <ch**********@hp.comwrites:
[1GB of what? The-here desktop machine has (fx:grope) 2Gb of memory; a one-
off 1Gb binary tree I could just read into a C program. You could just buy
more memory. (But remember who I work for ...)]
One of the places I work, I could file an Engineering Support
Services request online and in a few minutes a nice man would
show up with a handful of DIMMs and install them for me, no
questions asked. I imagine I could ask for 4 GB or so of RAM
before I'd need my manager to sign off on it.

But it's not normal, most of the time, to design software so that
it only runs efficiently with the latest and greatest and
highest-specced hardware. It's better, when you can, to make
your software work on the widest range of hardware that doesn't
require major algorithmic inconvenience.
--
Ben Pfaff
http://benpfaff.org
Mar 28 '07 #7
Ben Pfaff wrote:
Chris Dollin <ch**********@hp.comwrites:
>[1GB of what? The-here desktop machine has (fx:grope) 2Gb of memory; a one-
off 1Gb binary tree I could just read into a C program. You could just buy
more memory. (But remember who I work for ...)]

One of the places I work, I could file an Engineering Support
Services request online and in a few minutes a nice man would
show up with a handful of DIMMs and install them for me, no
questions asked. I imagine I could ask for 4 GB or so of RAM
before I'd need my manager to sign off on it.

But it's not normal, most of the time, to design software so that
it only runs efficiently with the latest and greatest and
highest-specced hardware. It's better, when you can, to make
your software work on the widest range of hardware that doesn't
require major algorithmic inconvenience.
I wasn't saying that the OP /should/ just go for more memory; just
pointing out that they're likely within the space of "do it all in
memory", if that's an available option. We don't know enough about
their context to know if it's realistic or not.

--
Yes, Virginia, there is a second Jena user conference: Palo Alto, Sep 2007.
"Anything can happen in the next half-hour." /Stingray/

Hewlett-Packard Limited registered office: Cain Road, Bracknell,
registered no: 690597 England Berks RG12 1HN

Mar 28 '07 #8
In article <11**********************@n59g2000hsh.googlegroups .com>,
Clement <je*******@gmail.comwrote:
>i want to create a binary tree[size of 1GB] so i want to place it in
Disk instead of memeory.. and i need to access through pointers.. can i
You could look to see if your system has functions for memory-mapping
a file (mmap() for example). But you will probably not be able to
guarantee that it is mapped at the same address in different runs of
the program, so it may well be more practical to use offsets rather
than pointers.

-- Richard
--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.
Mar 28 '07 #9
"Clement" <je*******@gmail.comwrote:

# i want to create a binary tree[size of 1GB] so i want to place it in
# Disk instead of memeory.. and i need to access through pointers.. can i

Use a memory mapped file. You can use pointer operations on the file
contents. Whether a file page is actually on disk or in memory becomes
the operating system concerns.

Alternatives are pure memory where you malloc nodes. You need to write
tree traversal functions to translate external representations and
in memory version; or to do your own paging by maintianing page frames
and using seeks, reads, and writes to shuffle active nodes in and out
of memory.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
A bunch of savages in this town.
Mar 28 '07 #10
SM Ryan wrote:
>
"Clement" <je*******@gmail.comwrote:

# i want to create a binary tree[size of 1GB] so i want to place it in
# Disk instead of memeory.. and i need to access through pointers.. can i

Use a memory mapped file. You can use pointer operations on the file
contents. Whether a file page is actually on disk or in memory becomes
the operating system concerns.
Note that memory-mapped files are beyond the scope of Standard C.
They are also system-specific, and not all systems support them.
If you do go that route, I would suggest at least putting wrappers
around the system-specific functions to allow easier porting to
other systems which support memory-mapped files differently.

[...]

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h|
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:Th*************@gmail.com>

Mar 28 '07 #11
On Mar 28, 1:38 pm, Kenneth Brody <kenbr...@spamcop.netwrote:
SM Ryan wrote:
"Clement" <jeba.r...@gmail.comwrote:
# i want to create a binary tree[size of 1GB] so i want to place it in
# Disk instead of memeory.. and i need to access through pointers.. can i
Use a memory mapped file. You can use pointer operations on the file
contents. Whether a file page is actually on disk or in memory becomes
the operating system concerns.

Note that memory-mapped files are beyond the scope of Standard C.
They are also system-specific, and not all systems support them.
If you do go that route, I would suggest at least putting wrappers
around the system-specific functions to allow easier porting to
other systems which support memory-mapped files differently.
Or find a toolkit that already does this.
I use ACE, but that is C++.
I seem to remember seeing similar things for C.

Mar 28 '07 #12
On Mar 28, 1:30 pm, SM Ryan <wyrm...@tango-sierra-oscar-foxtrot-
tango.fake.orgwrote:
"Clement" <jeba.r...@gmail.comwrote:

# i want to create a binary tree[size of 1GB] so i want to place it in
# Disk instead of memeory.. and i need to access through pointers.. can i

Use a memory mapped file. You can use pointer operations on the file
contents. Whether a file page is actually on disk or in memory becomes
the operating system concerns.

Alternatives are pure memory where you malloc nodes. You need to write
tree traversal functions to translate external representations and
in memory version; or to do your own paging by maintianing page frames
and using seeks, reads, and writes to shuffle active nodes in and out
of memory.
OR use an operating system that does page swapping for you. Modern
ones all do, so unless you are runinng under some dumb custom kernal
OS, you should be able to just let the OD do the swapping to disc.

Ed

Mar 28 '07 #13

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

32
by: John | last post by:
Hi all: When I run my code, I find that the memory that the code uses keeps increasing. I have a PC with 2G RAM running Debian linux. The code consumes 1.5G memory by the time it finishes...
6
by: Fred Zwarts | last post by:
Hello, I am trying to debug some complex debug code. In order to track the use of dynamically allocated memory, I replaced the standard global new and delete operators. (Not for changing the...
20
by: Jonas | last post by:
Hi, I'm 99 % sure that Standard C guarantees to do a memory move inside realloc() in case the new, returned memory block (address) is different than the original one. Can any C expert confirm...
8
by: vikram | last post by:
i have series of questions 1.How a c program is loaded in memory i mean the whats is the structure that the code segment?? data segment?? 2.When you say const int *p; where is p...
30
by: jimjim | last post by:
Hello, This is a simple question for you all, I guess . int main(){ double *g= new double; *g = 9; delete g; cout<< sizeof(g)<<" "<<sizeof(double)<<" "<<sizeof(*g)<<" "<<*g<<" "<<endl; *g =...
18
by: Ramasubbu Ramasubramanian XR (AS/EAB) | last post by:
What is memory leakage, could any one explain with sample code
25
by: Zeng | last post by:
I finally narrowed down my code to this situation, quite a few (not all) of my CMyClass objects got hold up after each run of this function via the simple webpage that shows NumberEd editbox. My...
2
by: Ashish | last post by:
hi all, I have been doing some performance testing of a asp_net website, to be hosted on a shared server .. as far as i understand every page when accessed first time is compiled and loaded in...
7
by: toton | last post by:
Hi, I have a STL vector of of characters and the character class has a Boost array of points. The things are vector<Characterchars; and class Character{ private: array<Point,Npoints; }; Now...
9
by: Hemal | last post by:
Hi All, I need to know the memory required by a c program. Is there any tool/utility which can give me the memory usage in terms of DATA segment, TEXT segment, BSS segment etc. I am working...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.