473,414 Members | 1,596 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,414 software developers and data experts.

general advice on data storage

Hi,

I'm new to vb.net and would be grateful if anyone could offer some
advice on the best way to approach this problem:

I wish to dynamically capture pricing information e.g. 10 stock prices
and use this data to generate a number of graphs over time (in real
time) of each share price. This means for each share I need to store a
price and a timestamp.

I would need to collect around 600 pairs of data per stock (once per
second for 10 mins), so around 6000 in total, then roll the data
forward so that I only ever keep 600 records i.e. always have the last
10 minutes worth of data for each stock.

I would therefore like to know the best method to store this data on
the fly. I can only think of an array or dataset as a way to do this
but maybe there are better solutions I am unaware of?

Thanks for any help,

Mike

Nov 21 '05 #1
5 1213
Mike,

It is a very rough answer for a newsgroup, however I think that I would
first try for an SQLExpress or the old MSDE database. With a table per
share.

Than add it everytime rows using the insert to the share tables with a new
datatime field as key.

Creating those tables is easy to do in VBNet in a for each loop.

To get than a datatable for the last 10 minutes is just getting that with a
fill with a "where" clause the < datatime. It is not slow however I don't
know if it is quick enough.

It needs of course consequent deleting with another seperate running program
from the old rows.

Using the dataset/datatable itself inside a dataset will almost for sure
break you up in VS2003. Deleting of that will probably be to slow (this
should be improved in VS2005, however I did not try it yet)

Just as first idea when I saw your question.

Cor
Nov 21 '05 #2
Thanks Cor - much appreciated. I'll try out your suggestions.

Nov 21 '05 #3
How often will the graph be redrawn?

In addition to Cor's reply, here's another option:
Use a linked list. Something like the following [psuedocode]:

public struct StockHistoryItem
Dim price as double
Dim timestamp as DateTime
end struct

class StockHistory
Private data() as StockHistoryItem
Private startData as Long = 0
Private endData as Long = 0
Private maxSize as Long = 0

public Sub New(byref size as Long)
if Size>0 then
maxSize = size
Redim data(maxSize) As StockHistoryItem
else
throw new Exception - invalid
end if
end sub

public sub AddItem(ByVal newItem as StockHistoryItem)
if endData = maxSize then
endData = 0
else
endData += 1
end if
data(endData) = newItem
if endData = maxSize Then
startData = 0
else
startData += 1
end if
end sub
end class

Bear in mind I don't think the upper bound stuff is correct - you might need
(endData + 1) in the "If" statements involving it. You obviously need
functions to retrieve data - you could implement IEnumerable or have
something like:
public sub GetItem(byref index as Long)
if index >=0 AND index <= maxsize - 1 then
' You also need to check in case you haven't yet filled the
structure with values.
GetItem = data(Dim item As Long = (index + startData) Mod
maxSize)
end if
end sub
If you took Cor's route, you don't necessarily need an entirely separate
program, you could clean up old rows each time you draw the graph.

Jevon
"Mike" <my***@pearcey2001.freeserve.co.uk> wrote in message
news:11**********************@g14g2000cwa.googlegr oups.com...
Hi,

I'm new to vb.net and would be grateful if anyone could offer some
advice on the best way to approach this problem:

I wish to dynamically capture pricing information e.g. 10 stock prices
and use this data to generate a number of graphs over time (in real
time) of each share price. This means for each share I need to store a
price and a timestamp.

I would need to collect around 600 pairs of data per stock (once per
second for 10 mins), so around 6000 in total, then roll the data
forward so that I only ever keep 600 records i.e. always have the last
10 minutes worth of data for each stock.

I would therefore like to know the best method to store this data on
the fly. I can only think of an array or dataset as a way to do this
but maybe there are better solutions I am unaware of?

Thanks for any help,

Mike

Nov 21 '05 #4
Sorry, I should add to this that it obviously isn't necessarily the best
option if memory is an issue, but for the amount of data you mention it
shouldn't be too much of a problem.

Jevon

"Jevon" <pl****@ask.com> wrote in message
news:ek**************@tk2msftngp13.phx.gbl...
How often will the graph be redrawn?

In addition to Cor's reply, here's another option:
Use a linked list. Something like the following [psuedocode]:

public struct StockHistoryItem
Dim price as double
Dim timestamp as DateTime
end struct

class StockHistory
Private data() as StockHistoryItem
Private startData as Long = 0
Private endData as Long = 0
Private maxSize as Long = 0

public Sub New(byref size as Long)
if Size>0 then
maxSize = size
Redim data(maxSize) As StockHistoryItem
else
throw new Exception - invalid
end if
end sub

public sub AddItem(ByVal newItem as StockHistoryItem)
if endData = maxSize then
endData = 0
else
endData += 1
end if
data(endData) = newItem
if endData = maxSize Then
startData = 0
else
startData += 1
end if
end sub
end class

Bear in mind I don't think the upper bound stuff is correct - you might
need (endData + 1) in the "If" statements involving it. You obviously need
functions to retrieve data - you could implement IEnumerable or have
something like:
public sub GetItem(byref index as Long)
if index >=0 AND index <= maxsize - 1 then
' You also need to check in case you haven't yet filled the
structure with values.
GetItem = data(Dim item As Long = (index + startData) Mod
maxSize)
end if
end sub
If you took Cor's route, you don't necessarily need an entirely separate
program, you could clean up old rows each time you draw the graph.

Jevon
"Mike" <my***@pearcey2001.freeserve.co.uk> wrote in message
news:11**********************@g14g2000cwa.googlegr oups.com...
Hi,

I'm new to vb.net and would be grateful if anyone could offer some
advice on the best way to approach this problem:

I wish to dynamically capture pricing information e.g. 10 stock prices
and use this data to generate a number of graphs over time (in real
time) of each share price. This means for each share I need to store a
price and a timestamp.

I would need to collect around 600 pairs of data per stock (once per
second for 10 mins), so around 6000 in total, then roll the data
forward so that I only ever keep 600 records i.e. always have the last
10 minutes worth of data for each stock.

I would therefore like to know the best method to store this data on
the fly. I can only think of an array or dataset as a way to do this
but maybe there are better solutions I am unaware of?

Thanks for any help,

Mike


Nov 21 '05 #5
Thanks for the alternative. I would ideally like to redraw the graph
each time a new data point is stored, so I'm hoping once a second.

I'm not sure whether the performance of graphing add-ins would be able
to handle this or not, but maybe I could resort to drawing it if they
can't. My ultimate aim would be to have each of these graphs running
within a datagrid cell (which would most likely require me to draw
manually) but that is way beyond my current level so I'll save that
part of it as a project for the future :-)

Thanks again,

Mike

Nov 21 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: SophistiCat | last post by:
Hi, I am working on a computational program that has to read a number of parameters (~50) from an input file. The program contains a single class hierarchy with about a dozen member-classes or...
3
by: Alex | last post by:
Hi all, I'm looking for some advice on how best to implement storage of access logs into a db/2 8.1.4 database running on a RH 7.2 system. I have 5 (squid) web caches running here that...
9
by: pankaj_wolfhunter | last post by:
Hi, I need some clearance on the following questions: 1) Does LOAD command updates indexes defined for a table? 2) Is REPLACE option in the LOAD command a logged operation? Help will be...
3
by: James Armstrong | last post by:
Hi all, (warning - long post ahead) I have been tasked with designing a database for my company which will store trade information (it is a financial firm). It will need to export this info...
7
by: Joe Ross | last post by:
I've been working with Microsoft support for over 3 weeks now on an intermittent General Network Error we're seeing in our production environment between our ASP.NET application and SQL Server...
3
by: Agnes | last post by:
My client hold its MS SQL server in hkbranch, Both china and hong kong office can run the vb.net application very well via VPN. Now, they want to reduce cost and move the MS SQL server to china 's...
4
by: va | last post by:
I realize that ASP.NET profile provider is used to create a strongly typed storage of data that can be used like Session variables but can survive beyond sessions. In my ASP.NET 2.0 application,...
37
by: dmoran21 | last post by:
I am a mathematician trying to write a program in C to do some curve fitting. When I get to the point where I attempt to enter data in my arrays, I get a General Protection Exception error message....
10
by: Les Desser | last post by:
In article <fcebdacd-2bd8-4d07-93a8-8b69d3452f3e@s50g2000hsb.googlegroups.com>, The Frog <Mr.Frog.to.you@googlemail.comMon, 14 Apr 2008 00:45:10 writes Not sure if I quite follow that. 1....
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.