473,698 Members | 2,022 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

general advice on data storage

Hi,

I'm new to vb.net and would be grateful if anyone could offer some
advice on the best way to approach this problem:

I wish to dynamically capture pricing information e.g. 10 stock prices
and use this data to generate a number of graphs over time (in real
time) of each share price. This means for each share I need to store a
price and a timestamp.

I would need to collect around 600 pairs of data per stock (once per
second for 10 mins), so around 6000 in total, then roll the data
forward so that I only ever keep 600 records i.e. always have the last
10 minutes worth of data for each stock.

I would therefore like to know the best method to store this data on
the fly. I can only think of an array or dataset as a way to do this
but maybe there are better solutions I am unaware of?

Thanks for any help,

Mike

Nov 21 '05 #1
5 1225
Mike,

It is a very rough answer for a newsgroup, however I think that I would
first try for an SQLExpress or the old MSDE database. With a table per
share.

Than add it everytime rows using the insert to the share tables with a new
datatime field as key.

Creating those tables is easy to do in VBNet in a for each loop.

To get than a datatable for the last 10 minutes is just getting that with a
fill with a "where" clause the < datatime. It is not slow however I don't
know if it is quick enough.

It needs of course consequent deleting with another seperate running program
from the old rows.

Using the dataset/datatable itself inside a dataset will almost for sure
break you up in VS2003. Deleting of that will probably be to slow (this
should be improved in VS2005, however I did not try it yet)

Just as first idea when I saw your question.

Cor
Nov 21 '05 #2
Thanks Cor - much appreciated. I'll try out your suggestions.

Nov 21 '05 #3
How often will the graph be redrawn?

In addition to Cor's reply, here's another option:
Use a linked list. Something like the following [psuedocode]:

public struct StockHistoryIte m
Dim price as double
Dim timestamp as DateTime
end struct

class StockHistory
Private data() as StockHistoryIte m
Private startData as Long = 0
Private endData as Long = 0
Private maxSize as Long = 0

public Sub New(byref size as Long)
if Size>0 then
maxSize = size
Redim data(maxSize) As StockHistoryIte m
else
throw new Exception - invalid
end if
end sub

public sub AddItem(ByVal newItem as StockHistoryIte m)
if endData = maxSize then
endData = 0
else
endData += 1
end if
data(endData) = newItem
if endData = maxSize Then
startData = 0
else
startData += 1
end if
end sub
end class

Bear in mind I don't think the upper bound stuff is correct - you might need
(endData + 1) in the "If" statements involving it. You obviously need
functions to retrieve data - you could implement IEnumerable or have
something like:
public sub GetItem(byref index as Long)
if index >=0 AND index <= maxsize - 1 then
' You also need to check in case you haven't yet filled the
structure with values.
GetItem = data(Dim item As Long = (index + startData) Mod
maxSize)
end if
end sub
If you took Cor's route, you don't necessarily need an entirely separate
program, you could clean up old rows each time you draw the graph.

Jevon
"Mike" <my***@pearcey2 001.freeserve.c o.uk> wrote in message
news:11******** **************@ g14g2000cwa.goo glegroups.com.. .
Hi,

I'm new to vb.net and would be grateful if anyone could offer some
advice on the best way to approach this problem:

I wish to dynamically capture pricing information e.g. 10 stock prices
and use this data to generate a number of graphs over time (in real
time) of each share price. This means for each share I need to store a
price and a timestamp.

I would need to collect around 600 pairs of data per stock (once per
second for 10 mins), so around 6000 in total, then roll the data
forward so that I only ever keep 600 records i.e. always have the last
10 minutes worth of data for each stock.

I would therefore like to know the best method to store this data on
the fly. I can only think of an array or dataset as a way to do this
but maybe there are better solutions I am unaware of?

Thanks for any help,

Mike

Nov 21 '05 #4
Sorry, I should add to this that it obviously isn't necessarily the best
option if memory is an issue, but for the amount of data you mention it
shouldn't be too much of a problem.

Jevon

"Jevon" <pl****@ask.com > wrote in message
news:ek******** ******@tk2msftn gp13.phx.gbl...
How often will the graph be redrawn?

In addition to Cor's reply, here's another option:
Use a linked list. Something like the following [psuedocode]:

public struct StockHistoryIte m
Dim price as double
Dim timestamp as DateTime
end struct

class StockHistory
Private data() as StockHistoryIte m
Private startData as Long = 0
Private endData as Long = 0
Private maxSize as Long = 0

public Sub New(byref size as Long)
if Size>0 then
maxSize = size
Redim data(maxSize) As StockHistoryIte m
else
throw new Exception - invalid
end if
end sub

public sub AddItem(ByVal newItem as StockHistoryIte m)
if endData = maxSize then
endData = 0
else
endData += 1
end if
data(endData) = newItem
if endData = maxSize Then
startData = 0
else
startData += 1
end if
end sub
end class

Bear in mind I don't think the upper bound stuff is correct - you might
need (endData + 1) in the "If" statements involving it. You obviously need
functions to retrieve data - you could implement IEnumerable or have
something like:
public sub GetItem(byref index as Long)
if index >=0 AND index <= maxsize - 1 then
' You also need to check in case you haven't yet filled the
structure with values.
GetItem = data(Dim item As Long = (index + startData) Mod
maxSize)
end if
end sub
If you took Cor's route, you don't necessarily need an entirely separate
program, you could clean up old rows each time you draw the graph.

Jevon
"Mike" <my***@pearcey2 001.freeserve.c o.uk> wrote in message
news:11******** **************@ g14g2000cwa.goo glegroups.com.. .
Hi,

I'm new to vb.net and would be grateful if anyone could offer some
advice on the best way to approach this problem:

I wish to dynamically capture pricing information e.g. 10 stock prices
and use this data to generate a number of graphs over time (in real
time) of each share price. This means for each share I need to store a
price and a timestamp.

I would need to collect around 600 pairs of data per stock (once per
second for 10 mins), so around 6000 in total, then roll the data
forward so that I only ever keep 600 records i.e. always have the last
10 minutes worth of data for each stock.

I would therefore like to know the best method to store this data on
the fly. I can only think of an array or dataset as a way to do this
but maybe there are better solutions I am unaware of?

Thanks for any help,

Mike


Nov 21 '05 #5
Thanks for the alternative. I would ideally like to redraw the graph
each time a new data point is stored, so I'm hoping once a second.

I'm not sure whether the performance of graphing add-ins would be able
to handle this or not, but maybe I could resort to drawing it if they
can't. My ultimate aim would be to have each of these graphs running
within a datagrid cell (which would most likely require me to draw
manually) but that is way beyond my current level so I'll save that
part of it as a project for the future :-)

Thanks again,

Mike

Nov 21 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
1990
by: SophistiCat | last post by:
Hi, I am working on a computational program that has to read a number of parameters (~50) from an input file. The program contains a single class hierarchy with about a dozen member-classes or inherited classes, each of which needs some subset of those input parameters. The classes may individually perform some input validation, and even determine which parameters are to be read next. Currently, each class performs its own file...
3
2067
by: Alex | last post by:
Hi all, I'm looking for some advice on how best to implement storage of access logs into a db/2 8.1.4 database running on a RH 7.2 system. I have 5 (squid) web caches running here that service the whole university. All access to external web sites must go through these caches. Each cache generates a gzip'd access log file that's about 100Mbytes every night.
9
1699
by: pankaj_wolfhunter | last post by:
Hi, I need some clearance on the following questions: 1) Does LOAD command updates indexes defined for a table? 2) Is REPLACE option in the LOAD command a logged operation? Help will be greatly appreciated. TIA
3
1667
by: James Armstrong | last post by:
Hi all, (warning - long post ahead) I have been tasked with designing a database for my company which will store trade information (it is a financial firm). It will need to export this info into an excel file while converting some of the data into an export format (example - we use B for buy, the firm we export to uses BY). Eventually, accounting will also need reports from the data.
7
2024
by: Joe Ross | last post by:
I've been working with Microsoft support for over 3 weeks now on an intermittent General Network Error we're seeing in our production environment between our ASP.NET application and SQL Server 2000. They are continuing to work on the issue, but it seems as if our progress is grinding to a halt. I asked this question on the newsgroups before going to MS but figured I'd give it another shot now that I have more information. The problem...
3
2223
by: Agnes | last post by:
My client hold its MS SQL server in hkbranch, Both china and hong kong office can run the vb.net application very well via VPN. Now, they want to reduce cost and move the MS SQL server to china 's data centre . During testing period, the user cannot connect to the SQL server and alwasy got "General Network Error", we said that the network connection may not be stable and suggest move back the SQL server. However, the Network Engineer of...
4
1354
by: va | last post by:
I realize that ASP.NET profile provider is used to create a strongly typed storage of data that can be used like Session variables but can survive beyond sessions. In my ASP.NET 2.0 application, I am thinking of using the ASP.NET profile provider and its the binary storage capability to store a lot of my user's custom variables versus rolling my own object storage and access in a database. Are there any limits or warnings I should be...
37
7160
by: dmoran21 | last post by:
I am a mathematician trying to write a program in C to do some curve fitting. When I get to the point where I attempt to enter data in my arrays, I get a General Protection Exception error message. What is this and how can I fix it. I am writing in Turbo C++. My source code is below. #include <stdio.h> #include <stdlib.h> #include <math.h>
10
3365
by: Les Desser | last post by:
In article <fcebdacd-2bd8-4d07-93a8-8b69d3452f3e@s50g2000hsb.googlegroups.com>, The Frog <Mr.Frog.to.you@googlemail.comMon, 14 Apr 2008 00:45:10 writes Not sure if I quite follow that. 1. Data encrypted by AES key 2. AES key encrypted with Asymmetric public key (?)
0
9157
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
8895
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8861
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7725
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6518
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4369
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4619
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3046
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2329
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.