What will be the best data structure to store the huge amount of the data and why?
4  2649   Ganon11 3,652
Recognized Expert Specialist
The 'best' data structure to use does not necessarily depend on the size of the data, but on what you need to do with it.
For example, if you need fast, random access to members, a vector might work. This has drawbacks, though, as a vector (I believe) stores its data in sequential order in memory, thus requiring millions of bytes of memory in a row to work.
If you don't need very fast access, or you need to be able to re-order and delete quickly, a linked list would work. Linked Lists don't need sequential memory - its data is stored randomly throughout memory - but it does require that you traverse through the whole list to find a certain element (or half the list at best, using a doubly-linked list).
If you need to keep these things in balance, I would suggest a binary search tree. The BST offers relatively quick access, relatively quick deletion, maintains sorted order, and does not require sequential order.
These are just 3 common data sets - you may also like to look at stacks, queues, AVL trees, sets, maps, etc.
If you have so much data that it won't fit in the memory, check out STXXL which implements hard disk caching.
You use Oracle or SQLServer or some equivalent product for huge amounts of data.
thanks every one for the responses.
Sign in to post your reply or Sign up for a free account.
Similar topics | |
by: Bob Bedford |
last post by:
Sorry if I post here, but I don't have access to any valuable mysql ng for
this question, ans since 99% or php programmers deal with mysql, I'm sure
I'll get an answer here.
Look at the folloming table:
CREATE TABLE searchlog (
IDSearchLog MEDIUMINT UNSIGNED NOT NULL auto_increment,
F1 MEDIUMINT UNSIGNED default NULL,
F2 VARCHAR(40) default NULL,
F3a SMALLINT UNSIGNED default NULL,
|
by: gordy |
last post by:
edit: this came out longer than I thought, any comments about anything
here is greatly appreciated. thank you for reading
My system stores millions of records, each with fields like firstname,
lastname, email address, city, state, zip, along with any number of user
defined fields. The application allows users to define message templates
with variables. They can then select a template, and for each variable
in the template, type in a...
|
by: narasimha |
last post by:
Hello,
I have a data record with two fields...<value, date>. I have more
than a million of these records. As I add more records into this set,
I want a data structure that would be efficient enough to find
duplicate records with the same value, and earliest possible date. It
should be efficient to insert the records into the data structure. Is
there any database structure that I can get help with ??
-Narasimha
|
by: VMI |
last post by:
We have this huge application that's based on storing tons of data on a
dataTable. The only problem we're having is that storing LOTS of data (1
million records) into a datatable will slow down the system since all this
data will be stored in memory. The performance is really affected with this.
Since we don't really want to redesign everything based on this datatable,
are there any companies that offer a product similar to the datatable...
|
by: hoopsho |
last post by:
Hi Everyone,
I am trying to write a program that does a few things very fast
and with efficient use of memory...
a) I need to parse a space-delimited file that is really large,
upwards fo a million lines.
b) I need to store the contents into a unique hash.
c) I need to then sort the data on a specific field.
d) I need to pull out certain fields and report them to the user.
| | |
by: Tommy Vercetti |
last post by:
I have an application that loads a 150 MB text file containing 6.6
million strings. I want to be able to quickly determine run lookups
against this and determine yes or no; is a given string in this collection.
This obviously requires a binary tree (or possibly a hash table). My
first solution was to use STL with std::set<std::string>. However, this
solution took up 600MB and was unacceptable. I revised this by reading
the entire 150MB...
|
by: James |
last post by:
I am using vb.net and need to keep in memory a large data structure, so I am
looking for the best option. And after several test I am pretty confused. So
I will be grateful if anyone can help me.
My basic need is:
|
by: Mudcat |
last post by:
Hi,
I am trying to build a tool that analyzes stock data. Therefore I am
going to download and store quite a vast amount of it. Just for a
general number - assuming there are about 7000 listed stocks on the two
major markets plus some extras, 255 tradying days a year for 20 years,
that is about 36 million entries.
Obviously a database is a logical choice for that. However I've never
used one, nor do I know what benefits I would get...
|
by: pereges |
last post by:
Hi, I'm in a bit of dilemma here.
I want to use an adjacency matrix for building edge lists in my
project.
The thing is an adjacency matrix will have nv * nv elements where nv
is number of vertices. Advantage is that it makes things very fast in
my project, only one pass over list of triangles as opposed to two.
But I'm a little skeptical about allocating memory for nv * nv
elements. nv can be very high up to 10,000,000 as well. ...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
| | |
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| | |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |
