473,383 Members | 1,813 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,383 software developers and data experts.

Need help extending stl list.

I have an stl list that grows to be too huge to maintain effectivly in
memory.
There are elements within the list that could be stored on disk until
accessed.

However I don't want to expose this to the application class.

How can I extent the stl list to write some elements to disk when they
are put in the list
and read them from disk when they are read from the list.

All the list methods need to still work like iterators and sort
methods....

TIA.
B.


Jul 22 '05 #1
6 1617

"JustSomeGuy" <No***@ucalgary.ca> wrote in message
news:40**************@ucalgary.ca...
I have an stl list that grows to be too huge to maintain effectivly in
memory.
There are elements within the list that could be stored on disk until
accessed.

However I don't want to expose this to the application class.

How can I extent the stl list to write some elements to disk when they
are put in the list
and read them from disk when they are read from the list.

All the list methods need to still work like iterators and sort
methods....


See GoF Proxy Pattern.

Jeff F
Jul 22 '05 #2
Jeff Flinn wrote:
"JustSomeGuy" <No***@ucalgary.ca> wrote in message
news:40**************@ucalgary.ca...
I have an stl list that grows to be too huge to maintain effectivly in
memory.
There are elements within the list that could be stored on disk until
accessed.

However I don't want to expose this to the application class.

How can I extent the stl list to write some elements to disk when they
are put in the list
and read them from disk when they are read from the list.

All the list methods need to still work like iterators and sort
methods....


See GoF Proxy Pattern.

Jeff F


Sorry I don't have this book... Nor do I know what you mean... But thanks
I'll try to figure out what you are trying to tell me...
Jul 22 '05 #3
Jeff Flinn wrote:
"JustSomeGuy" <No***@ucalgary.ca> wrote in message
news:40**************@ucalgary.ca...
I have an stl list that grows to be too huge to maintain effectivly in
memory.
There are elements within the list that could be stored on disk until
accessed.

However I don't want to expose this to the application class.

How can I extent the stl list to write some elements to disk when they
are put in the list
and read them from disk when they are read from the list.

All the list methods need to still work like iterators and sort
methods....


See GoF Proxy Pattern.

Jeff F


Sorry I don't have this book... Nor do I know what you mean... But thanks
I'll try to figure out what you are trying to tell me...


"Design Patterns" by Gamma, Helm, Johnson, Vlissides ( the Gang of Four )

I assume you are specifying a list because you need to do some constant time
insertions at arbitrary locations within the list. If not provide some more
detail, and there may be better approaches. You wouldn't extend std::list,
in
that you wouldn't derive from it. You don't provide much detail. But if it
is
costly to construct each item, then you would create a lightweight proxy
class
that represents each item.

class proxy
{
some_handle mHdl;
bool mLoaded;

void LoadIt(){ if( !mLoaded ){ /* load from disk using the handle
data*/ } }

public:

proxy( some_hdl aHdl ):mHdl(aHdl),mLoaded(false){}

void some_method(){ LoadIt(); /* some other actions */ }
}

typedef std::list<proxy> tProxyList;

some_func()
{
tProxyList lPs;

lPs.push_back( 1 );
lPs.push_back( 2 );

...

lPs.back().some_method(); // only this one is actually loaded

}

Jeff F
Jul 22 '05 #4

"JustSomeGuy" <No***@ucalgary.ca> wrote in message
news:40***************@ucalgary.ca...
Jeff Flinn wrote:
"JustSomeGuy" <No***@ucalgary.ca> wrote in message
news:40**************@ucalgary.ca...
I have an stl list that grows to be too huge to maintain effectivly in
memory.
There are elements within the list that could be stored on disk until
accessed.

However I don't want to expose this to the application class.

How can I extent the stl list to write some elements to disk when they
are put in the list
and read them from disk when they are read from the list.

All the list methods need to still work like iterators and sort
methods....


See GoF Proxy Pattern.

Jeff F


Sorry I don't have this book... Nor do I know what you mean... But thanks
I'll try to figure out what you are trying to tell me...


I don't have the book to hand either but I think what Jeff is saying is that
you shouldn't be trying to extend std::list. Instead you should use a
regular list but in that list you should store objects which are proxies for
your real objects.

The real objects maybe on disk, they may be in memory, the proxies will
handle that. But that is where you should be adding the smart behaviour to
your code, in the proxy objects not in the list.

For instance the proxy class could maintain a pool of frequently accessed
objects, those would be stored in memory and the remainder on disk. Some
algorithm could determine when an object counts as frequently accessed and
is added to the pool (and when a less frequently accessed object is removed
from the pool).

john
john
Jul 22 '05 #5

"Jeff Flinn" <NO****@nowhere.com> wrote in message
news:c9**********@bluegill.adi.com...
Jeff Flinn wrote:
"JustSomeGuy" <No***@ucalgary.ca> wrote in message
news:40**************@ucalgary.ca...
> I have an stl list that grows to be too huge to maintain effectivly in > memory.
> There are elements within the list that could be stored on disk until > accessed.
>
> However I don't want to expose this to the application class.
>
> How can I extent the stl list to write some elements to disk when they > are put in the list
> and read them from disk when they are read from the list.
>
> All the list methods need to still work like iterators and sort
> methods....

See GoF Proxy Pattern.

Jeff F
Sorry I don't have this book... Nor do I know what you mean... But thanks I'll try to figure out what you are trying to tell me...


"Design Patterns" by Gamma, Helm, Johnson, Vlissides ( the Gang of Four )

I assume you are specifying a list because you need to do some constant

time insertions at arbitrary locations within the list. If not provide some more detail, and there may be better approaches. You wouldn't extend std::list,
in
that you wouldn't derive from it. You don't provide much detail. But if it
is
costly to construct each item, then you would create a lightweight proxy
class
that represents each item.

class proxy
{
some_handle mHdl;
bool mLoaded;

void LoadIt(){ if( !mLoaded ){ /* load from disk using the handle
data*/ } }

public:

proxy( some_hdl aHdl ):mHdl(aHdl),mLoaded(false){}

void some_method(){ LoadIt(); /* some other actions */ }
}

typedef std::list<proxy> tProxyList;

some_func()
{
tProxyList lPs;

lPs.push_back( 1 );
lPs.push_back( 2 );

...

lPs.back().some_method(); // only this one is actually loaded

}

Jeff F


I think you are on the right track and understand my problem.
My problem is that some objects are simply too large to be maintained in
memory
effectivly.. yet the requirements that they be orginized as a list is sound.
I was thinking that over-riding operator= for the class of objects in the
list might
be an easy way to impliment this.. However I guess there is no way to tell
if I actually
need the data or the handle to it. Adding a 'get' method is what I believe
you are suggesting,
but that changes the class interface doesn't it?

Jul 22 '05 #6
John Harrison wrote:
"JustSomeGuy" <No***@ucalgary.ca> wrote in message
news:40***************@ucalgary.ca...
Jeff Flinn wrote:
"JustSomeGuy" <No***@ucalgary.ca> wrote in message
news:40**************@ucalgary.ca...
> I have an stl list that grows to be too huge to maintain effectivly in
> memory.
> There are elements within the list that could be stored on disk until
> accessed.
>
> However I don't want to expose this to the application class.
>
> How can I extent the stl list to write some elements to disk when they
> are put in the list
> and read them from disk when they are read from the list.
>
> All the list methods need to still work like iterators and sort
> methods....

See GoF Proxy Pattern.

Jeff F


Sorry I don't have this book... Nor do I know what you mean... But thanks
I'll try to figure out what you are trying to tell me...


I don't have the book to hand either but I think what Jeff is saying is that
you shouldn't be trying to extend std::list. Instead you should use a
regular list but in that list you should store objects which are proxies for
your real objects.

The real objects maybe on disk, they may be in memory, the proxies will
handle that. But that is where you should be adding the smart behaviour to
your code, in the proxy objects not in the list.

For instance the proxy class could maintain a pool of frequently accessed
objects, those would be stored in memory and the remainder on disk. Some
algorithm could determine when an object counts as frequently accessed and
is added to the pool (and when a less frequently accessed object is removed
from the pool).

john

john


Sorry my design pattern knowlege is slim, I found the uml representation but
I'm not sure
how this helps me.
However if I understand you, then the list contains instances of the Proxy
class not
instances of the actual class. Is that correct?

Jul 22 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Alicia Haumann | last post by:
I accidentally sent this to webmaster@python.org, so this could be a duplicate if "webmaster" forwards it to this list. :{ Hi, there. Thanks for any help that can be offered. I've been...
3
by: new_GUY | last post by:
I have a HUGE project (at least for me) and need some guidance. I am trying to create a database for a local university movie club that allows users to input there basic personal information...
0
by: Brian O'Brien | last post by:
I have an stl list that grows to be too huge to maintain effectivly in memory. There are elements within the list that could be stored on disk until accessed. However I don't want to expose...
20
by: Keith Elder | last post by:
I ran into a unique situation today whereby I have a core library that uses generics to return users from Active Directory. Example: List<ADUser> users = ADUser.GetByName("First", "Last"); ...
5
by: vbgunz | last post by:
Hello everyone. I own two books. Learning Python and Python in a nutshell. When cross referencing the two books to try and clarify the ideas behind extending methods and delegates, this is where...
1
by: jeremito | last post by:
I am trying to learn how to extend and/or embed Python. I have looked at the document "Extending and Embedding the Python Interpreter" and also "Python/C API Reference Manual. In the examples...
3
by: Jeff | last post by:
Hey ASP.NET 2.0 I'm trying to extend the MembershipUser class, and have encounter a problem: << See in the middle of this post for info about why I do this >> << See below of this post for...
15
by: rhino | last post by:
I've put together a prototype of two-tiered CSS tabs that works really well in IE6, IE7, and FF2. It also works very well in Opera 9.27 _except_ that the placement of the lower tier of tabs is...
6
by: shapper | last post by:
Hello, I am creating a form that includes a few JQuery scripts and TinyMCE Editor: http://www.27lamps.com/Beta/Form/Form.html I am having a few problems with my CSS: 1. Restyling the Select
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.