473,320 Members | 1,854 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Best way to read in a lot of data, manipulate it, and write it.

I'm doing work with GPS and I have a file thats around 4Mb. It contains GPS information and my job is detect, correct, and output the results to another file which is in the same format. The file has around 30000 lines of text.

My problem is I'm reading in the data from the file into an array.
char array[30000][180];

I'm on a windows system and the first size of 30000 is too large. When I run the program, it crashes when creating that variable. 10,000 wont crash, but 10,000 obviously wont work for what I have to do.

So, ideally, how I do read just one line, do what I need to do to the data, and write it or store it and move on to the next line?
Nov 27 '06 #1
1 1178
horace1
1,510 Expert 1GB
I'm doing work with GPS and I have a file thats around 4Mb. It contains GPS information and my job is detect, correct, and output the results to another file which is in the same format. The file has around 30000 lines of text.

My problem is I'm reading in the data from the file into an array.
char array[30000][180];

I'm on a windows system and the first size of 30000 is too large. When I run the program, it crashes when creating that variable. 10,000 wont crash, but 10,000 obviously wont work for what I have to do.

So, ideally, how I do read just one line, do what I need to do to the data, and write it or store it and move on to the next line?
did you define the 30000 * 180 char array as a local variable defined inside a function?
try it as an external variable - it should work Ok with a modern compiler.
Nov 27 '06 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

1
by: rishka | last post by:
Rishka Mar 17, 5:40 am show options Newsgroups: comp.databases.oracle.tools From: "Rishka" <ris...@webmail.co.za> - Find messages by this author Date: 17 Mar 2005 05:40:45 -0800 Local:...
16
by: D Witherspoon | last post by:
I am developing a Windows Forms application in VB.NET that will use .NET remoting to access the data tier classes. A very simple way I have come up with is by creating typed (.xsd) datasets. For...
7
by: TechBoy | last post by:
RE: Access 2002 SP3 I have an app "App1.mdb" that has internal security built-in via a table: tblSecurity. In this table a match is obtained to see if they are authorized to use the app. This...
2
by: STiAT | last post by:
Hello. I tried to find a similar Function to WriteMemory, and couldnt find any memory manipulating functions in dotNET at all. I now wanted to know if there are any similar functions to the...
8
by: John | last post by:
Hi, I have an application that receives data (floats) via a socket. My job is to receive the bytes, byte-swap them and convert them to floats, and make them available to the application. The...
9
by: Luke Vogel | last post by:
Hi all. This is a bit of a newbie type question. I am trying to figure out what is the best way to connect to a database; ado.net, odic others? I've found a couple of examples that show you...
3
by: nicolasg | last post by:
Hi, I'm trying to open a file (any file) in binary mode and save it inside a new text file. After that I want to read the source from the text file and save it back to the disk with its...
16
by: Rex | last post by:
Hi All - I have a question that I think MIGHT be of interest to a number of us developers. I am somewhat new to VIsual Studio 2005 but not new to VB. I am looking for ideas about quick and...
0
by: kalichakradhar | last post by:
hi, I am new to VB programming.. I am writing a automated Tool in Vb which could read/write cell value present in the MPP (project plan 2003).I am successful in opening the project but now i dont...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.