473,387 Members | 1,463 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

Loading huge files

m6s
55
Hello people,

I have been given an assignment to filter out lines from various excel files and then do some transactions to a db.

I have the question how to load the lines from an excel, so as to use in the best way memory.
Assume that I could have 4 lines, 40, or...40000 lines. And for each line I would do a seek in a database and then some insert and have reports on screen.

Should I make an arraylist? Use some dataset from datatables where will each line as record? Some other solution?

The point is not to have a hitback on the machine...

Thank you in advance...
Dimitris
Nov 23 '07 #1
0 703

Sign in to post your reply or Sign up for a free account.

Similar topics

4
by: Ram | last post by:
We are trying to load text tab delimited files into SQL server using informatica 5.1 version. The text files are FTPed to a local server and then files are moved to applicable folder before...
5
by: Manish Jain | last post by:
Environment: Windows 2000 Server (SP4), ASP.Net/C# (Framework 1.1) Hardware: PIII @ 700 MHz, 512 MB RAM ----------------------------------------------------------------------------...
7
by: Staale L. Hansen | last post by:
We have an application which uses some mixed-mode code to expose a .NET 1.1 managed API. Only the necessary files are compiled with /clr. We want to be able to load the application without .NET...
5
by: Pete Marsh | last post by:
Wondering if anyone can recomend some sample code for dynamically loading the GD module. I have tried setting the extension dir in php.ini, and loading the GD module from there when apache is...
5
by: yeoj13 | last post by:
Hello, I have a db2load script I'm using to populate a large table. Ideally, my target table is required to have "Not Null" constraints on a number of different columns. I've noticed a ...
5
by: henryrhenryr | last post by:
Appologies for unclear title. I don't really understand the problem I'm facing. My system: PHP 5.2.1, Win2K, Apache 2, MySQL 5 (local - problem is same on my live site running php 4.3.9, Linux,...
2
by: Fromethius | last post by:
All week now I have been searching for a good way to load and display PNG files. Every solution I try never works. So now I am posting on the forums. I am not looking for a huge library. In fact,...
4
by: =?Utf-8?B?UGhpbCBKb2huc29u?= | last post by:
Hello, I am working on a peice of code that takes a file and loads it into a SQL Server DB. When I load large files (over 70Mb) I get a system out of memory exception. My machine has 2G RAM ...
5
by: s0suk3 | last post by:
Hi, I wanted to know how cautious it is to do something like: f = file("filename", "rb") f.read() for a possibly huge file. When calling f.read(), and not doing anything with the return...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.