473,746 Members | 2,222 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

how to automate the data load

ddtpmyra
333 Contributor
Hi!
I dont have any scripts to add right now. But I need help to give me idea how to do automated one. I used to do it on Oracle on Unix using .bat file and calling the text file containing my sqlloader command.

Now Im facing MS SQL Server 2003 ofcourse on Windows. Anybody can share a thoughts on this or a sample step by step on how to do it?

thanks!
Oct 26 '10 #1
3 2028
ck9663
2,878 Recognized Expert Specialist
There are a lot of factors to consider regarding your question.

1. Frequency of execution. Since you're asking about automation, I assume this will be executed more than once, in some defined frequency.

2. File size.

3. File type.

There are a number of options to automate your process.

1. Develop an external module (VB, Java, C, etc). Parse the file, connect to db and insert record. Use a scheduler if necessary.

2. Create a package in SSIS. Create a job to call the package.

3. Script it. Create T-SQLs, create a job to call the SP.

4. Buy an Integration Software, aka ETL Tool.

Good Luck!!!

~~ CK
Oct 26 '10 #2
ddtpmyra
333 Contributor
CK,
do you have sample code on VB?
Oct 27 '10 #3
ck9663
2,878 Recognized Expert Specialist
No. Try posting this on the VB/ASP/.Net forum. They might be able to help you more...

Good Luck!!!

~~ CK
Oct 27 '10 #4

Sign in to post your reply or Sign up for a free account.

Similar topics

0
3646
by: Harshal | last post by:
Hi all, we are trying to load Korean data from a Excel Sheet in to Oracle Database (8.1.7.3 with NLS_LANG=American_America.UTF8) We successfully transferred the Unicode file to our Unix server... now on server side, what should we set to NLS_LANG so that when we perform dataload via sqlloader, it will be in right format and when u spool the select on the recently loaded data it will be Korean data.. not junk Chars like Boxes and...
1
1804
by: Catch_22 | last post by:
Hi, I'm extracting data from a mainframe application with a view to loading it into a MS SQL database. I'm trying to determine the most efficient way to format the mainframe extract file to make loading into the database easier. The problem I have is that the existing record structure includes an array that can vary between 1 to 50. If I include this array in a single record the table I use to import the data would need 50 columns
4
1385
by: Frank | last post by:
I am loading data through a OleDbDatareader. The load is started from a form by the user, but the actual load is done in a separate class, which is accessed from my form. Now, I want the user to see how the load is progressing, so I have a counter textbox, which I have passed to the load object as a texbox (ByRef). For every 100 records that are loaded, I update the Text attribute for the textbox with a new number. However, the textbox...
6
1226
by: Frank | last post by:
I have made an application that loads data through an OleDbDatareader. I would like the user to be able to interupt the load. Now, I could of course include an interrupt button in the form. However, once the load starts program excution will be in the object of my dataload class. And so I suspect that the users interaction with an interrupt button (which could set a flag in the load object) will not be handled.
0
1056
by: asheesh.rastogi | last post by:
Hi, I am running DB2 UDB 8.2.4 on AIX 5.3. We are in process of migrating huge chunks of data as part of benchmarking exercise. I need to collect statistics like CPU time and total time taken to load data. We are using DB2 LOAD to load data into tables. I have shifted through the DB2 manual but all I can find is the -v option of the CLP, which only gives me the total time taken to load the data. Is there anyway to capture CPU time...
2
8720
by: Rick Shaw | last post by:
Hi, I have a problem with the datagridview not refreshed when the application first appear on the screen. The datagridview display data from a table in the dataset. At the same time, I've added checkbox columns that are not bounded the table. This datagridview is located in the tab (2nd). I thought I mention that since that might be part of the problem (?). When the applications start, it will need some parameter criteria selection...
0
1544
by: akp123 | last post by:
Hi I have a flat file with BLOB data that I want to load into DB2 running on AS400 V5R3. I tried to use iSeries navigator import function but it gives me a warning saying I don't have the reference file for the BLOB column. I am lost as to how to specify the filel path for the command CPYFRMIMPF to load the BLOB column. Any help is much appreciated.
0
1826
by: akp123 | last post by:
Hi I was wondering if anybody knows how to use CPYFRMIMPF command to load BLOB data from a flat file.
5
13916
by: orabalu | last post by:
Hi Guys, Can you give me some examples for Incremental load in PL/SQL for Datawarehouse projects. Regards, Balu
10
6381
by: sai1001 | last post by:
Hi, my job is converting informix queries to db2. There is a load function in informix to load the data from flat file to sql file. Can you please any body help how load function works in db2. i am using db2 v8.1 version. please help anybody, its urgent.
0
8970
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9486
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
9279
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9214
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8219
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
6057
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
1
3287
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2759
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2195
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.