473,383 Members | 1,861 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,383 software developers and data experts.

Best data structure to store the million of the data.

What will be the best data structure to store the huge amount of the data and why?
Jan 30 '08 #1
4 2636
Ganon11
3,652 Expert 2GB
The 'best' data structure to use does not necessarily depend on the size of the data, but on what you need to do with it.

For example, if you need fast, random access to members, a vector might work. This has drawbacks, though, as a vector (I believe) stores its data in sequential order in memory, thus requiring millions of bytes of memory in a row to work.

If you don't need very fast access, or you need to be able to re-order and delete quickly, a linked list would work. Linked Lists don't need sequential memory - its data is stored randomly throughout memory - but it does require that you traverse through the whole list to find a certain element (or half the list at best, using a doubly-linked list).

If you need to keep these things in balance, I would suggest a binary search tree. The BST offers relatively quick access, relatively quick deletion, maintains sorted order, and does not require sequential order.

These are just 3 common data sets - you may also like to look at stacks, queues, AVL trees, sets, maps, etc.
Jan 30 '08 #2
arnaudk
424 256MB
If you have so much data that it won't fit in the memory, check out STXXL which implements hard disk caching.
Jan 30 '08 #3
weaknessforcats
9,208 Expert Mod 8TB
You use Oracle or SQLServer or some equivalent product for huge amounts of data.
Jan 31 '08 #4
thanks every one for the responses.
Jan 31 '08 #5

Sign in to post your reply or Sign up for a free account.

Similar topics

10
by: Bob Bedford | last post by:
Sorry if I post here, but I don't have access to any valuable mysql ng for this question, ans since 99% or php programmers deal with mysql, I'm sure I'll get an answer here. Look at the...
5
by: gordy | last post by:
edit: this came out longer than I thought, any comments about anything here is greatly appreciated. thank you for reading My system stores millions of records, each with fields like firstname,...
3
by: narasimha | last post by:
Hello, I have a data record with two fields...<value, date>. I have more than a million of these records. As I add more records into this set, I want a data structure that would be efficient...
9
by: VMI | last post by:
We have this huge application that's based on storing tons of data on a dataTable. The only problem we're having is that storing LOTS of data (1 million records) into a datatable will slow down the...
11
by: hoopsho | last post by:
Hi Everyone, I am trying to write a program that does a few things very fast and with efficient use of memory... a) I need to parse a space-delimited file that is really large, upwards fo a...
3
by: Tommy Vercetti | last post by:
I have an application that loads a 150 MB text file containing 6.6 million strings. I want to be able to quickly determine run lookups against this and determine yes or no; is a given string in...
6
by: James | last post by:
I am using vb.net and need to keep in memory a large data structure, so I am looking for the best option. And after several test I am pretty confused. So I will be grateful if anyone can help me. ...
6
by: Mudcat | last post by:
Hi, I am trying to build a tool that analyzes stock data. Therefore I am going to download and store quite a vast amount of it. Just for a general number - assuming there are about 7000 listed...
13
by: pereges | last post by:
Hi, I'm in a bit of dilemma here. I want to use an adjacency matrix for building edge lists in my project. The thing is an adjacency matrix will have nv * nv elements where nv is number of...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.