473,322 Members | 1,409 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Design problem: dealing with multiple time frames.

424 256MB
I am trying to come up with a class design to deal with asynchronous data to be stored and analyzed over multiple time frames and could really use some design input. This is a rather long question but seeing the slowdown in the number of postings, I'm hoping some of you will have more time. (And this is not some kind of coursework question!)

I obtain data elements from an asynchronous data source (i.e. the data arrives at irregular time intervals) and wish to place them in a container of fixed length in time, which I call a "DataChart". This will be implemented using a deque. As a new data element arrives from the source, I remove all elements that are, say, 10mins older than the new data element from the back of the deque before pushing the new element on the front. This "10min DataChart" is my shortest time frame.

When the deque fills (i.e. the oldest DataElement is 10mins old and is about to be discarded as a new element arrives), I take a snapshot of the deque by averaging over its contents and store this average in another container of fixed width in time, a "1 day AveragesChart" which is implemented as a deque of 10min-AverageElements. Every 10mins the contents of the shortest timeframe, 10min-DataChart, are renewed and thus a new 10min-average element is generated to be stored in the 1-day-chart AveragesChart.

Likewise, I have a 1 month chart containing "1-day averages" made by averaging over the 1-day chart of 10min-averages when its contents are renewed once a day, and so on.

So far, I have a base class Chart which is the interface, and two derived classes DataChart and AveragesChart, each of which contain a deque and methods to add new data, etc.

Then I have a class ChartSet which holds all the charts, namely, the 10-min DataChart, the 1-day AveragesChart, the 1-month AveragesChart, etc, in its vector of Chart base handles.

In this way, I can have any number of time frames spanning arbitrary lengths in time, the shortest will always be a DataChart comprised of DataElements, and subsequent time frames AveragesCharts comprised of AverageElements.

So here are my questions,
  1. I have a pure virtual method addelement() in the base class, Chart, to update a given chart with new data. The overloaded addelement() in the derived DataChart takes DataElements. By contrast, addelement() in the derived AveragesChart takes AverageElements. This is my first dilema.
    I could make a base class "Element" to shoehorn derived DataElement and AverageElement objects into the base class addelement() method which can now just take generic Elements. But a DataElement and AverageElement objects are fundamentally different so I'd like to avoid this, plus I'd have to rewrite other portions related to the elements themselves, etc. It would be great if I could overload the base class pure virtual function like addelement(void) in a derived class as addelement(DataElement& data) but then the compiler complains that I haven't supplied an implementation of addelement(void) in derived classes. Your thoughts on this?
  2. Later, I'd like to compute other quantities from the charts, like a discrete Fourier transform, for example. I can add methods begin() and end() to the chart interface which return iterators to the Deques they contain and use those iterators in computations. Do you think this is a good idea? Does it expose my implementation of the Charts?
    Another way would be to have something like a visitor, or pass function objects to the charts containing the algorithm to be computed on the elements. I'm not sure what the design of that should be yet. Is that a good, extensible design?
  3. Which class do you think should have the responsibility of generating anAverageElement for the longer timeframe Chart when a shorter time frame Chart fills, the ChartSet object which encapsulates the Charts, or the Charts themselves?

Any other comments would be greatly appreciated.
May 31 '08 #1
4 2274
arnaudk
424 256MB
Is the question too long-winded?
Jun 16 '08 #2
Laharl
849 Expert 512MB
I can answer a couple small parts of this, though the whole of it...not a clue. I also think the ChartSet class should generate the new charts rather than the Charts themselves.

You can always define addelement(void) with
Expand|Select|Wrap|Line Numbers
  1. addelement(void){}
. I'm not sure how well that would work for you, though.
Jun 16 '08 #3
RedSon
5,000 Expert 4TB
Its not to long-winded, it is too technical. You discuss many implementation details that are irrelevant to the problem. What you need to do is explain your problem domain and then explain from a high level what your designed solution is (skipping the implementation details), then you might get more responses.

I think you are trying to have multiple queues then for each queue you average the data in one queue and then stick that value into another queue?

It seems like it might be better to just have one larger queue say data collected over 30 days then average for different time values.
Jun 16 '08 #4
arnaudk
424 256MB
I also think the ChartSet class should generate the new charts rather than the Charts themselves.
Yes, I agree that does make more sense.
You can always define addelement(void) with
Expand|Select|Wrap|Line Numbers
  1. addelement(void){}
.
Sure, that will get around the problem but it kind of defeats the purpose of having an abstract base class as an interface; if the interface contains an addelement() method, that should force derived classes to have a meaningful addelement(Element&) method. By adding an addelement(void) method in derived classes I effectively circumvent this requirement.

I'm starting to think my class model is incorrect: I have two unrelated types of elements which I want to hold in two related types of chart, one for each element type. This seems contradictory - for consistency, it seems that either
  1. the charts should be unrelated through inheritance, or
  2. the elements should be related through inheritance.
The problem with (a) is that I'd like to specify a base class interface for the charts which necessarily relates charts through inheritance, and the problem with (b) is that the elements are radically different and have very different methods and members so a base class for those would be kind of a hack. That's one of my dilemmas.
I think you are trying to have multiple queues then for each queue you average the data in one queue and then stick that value into another queue?
Yes, that's one aspect of what I'm trying to do.
It seems like it might be better to just have one larger queue say data collected over 30 days then average for different time values.
That was my initial idea. But it would be prohibitive insofar as memory is concerned because of the prodigious amount of data I have (spanning several years). In fact, I am concerned with the impact of the past on the present and future, so I only need a vague average of data in the far distant past since the precise details will have very little impact on what is happening in the present compared to the recent past (where I have shorter averages) or the very recent past (where I don't examine averages but the actual data elements instead). This fact saves me a lot of memory.
Jun 17 '08 #5

Sign in to post your reply or Sign up for a free account.

Similar topics

4
by: JesusFreak | last post by:
From: us_traveller@yahoo.com (JesusFreak) Newsgroups: microsoft.public.scripting.jscript Subject: toolbar script problem NNTP-Posting-Host: 192.92.126.136 Recently, I downloaded the following...
7
by: Pat Sheen | last post by:
I'm a novice at designing web pages so I'm not sure how complicated this is. I'd like a page that helps me take notes for research. It would have something like two frames. One with a word...
0
by: BlueMonkMN | last post by:
I've been trying to think of the right way to design relationships between objects with different desired lifetimes that raise events. If an event source is a relatively permanent object and the...
2
by: malcolm | last post by:
Hello, We have a robust (.NET 1.1 c# winforms) client-server application that utilizes many typed DataSets, typed DataTables and typed DataRows. Our application is a series of windows and popup...
0
by: frankenberry | last post by:
I have multi-page tiff files. I need to extract individual frames from the multi-page tiffs and save them as single-page tiffs. 95% of the time I receive multi-page tiffs containing 1 or more black...
3
by: Jan Ebbe Jensen | last post by:
Hi I have tried the following code. It works in Mozilla. In IE I'm not able to enable DesignMode? What have I done wrong? It says that obj is undefined? Can anyone help me please.
4
by: scottrm | last post by:
I am fairly new to oo design and I am looking at developing an object oriented asp.net application which will be built on top of a relational database. I have read quite a bit of the theory but...
29
by: MP | last post by:
Greets, context: vb6/ado/.mdb/jet 4.0 (no access)/sql beginning learner, first database, planning stages (I think the underlying question here is whether to normalize or not to normalize this...
5
by: istillshine | last post by:
Particularly for medium-sized (10,000 ~ 20,000 lines) programs, what are useful strategies to design them before coding? My strategies are: 1. Imagine what the final program would look like....
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.