By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
432,536 Members | 1,800 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 432,536 IT Pros & Developers. It's quick & easy.

How to speed up the import of 400,000+ records

P: n/a
jj
It's taking forever to upload 400,000 records to the database through
access/odbc, and I've tried phpMyAdmin's interface but it seems to timeout
during import of a CSV file. Is there a better way to import 400,000+
records?

Jul 19 '05 #1
Share this Question
Share on Google+
7 Replies


P: n/a
jj
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it like
you said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:
It's taking forever to upload 400,000 records to the database through
access/odbc, and I've tried phpMyAdmin's interface but it seems to timeout during import of a CSV file. Is there a better way to import 400,000+
records?


If it is possible for you to get the data into sql command-form like this:

-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt

Jul 19 '05 #2

P: n/a
Don't listen to the previous guy.

Use LOAD DATA CONCURRENT INFILE

Loading of several megs of data will take only a few seconds.

On Fri, 19 Dec 2003 19:30:04 GMT, "jj" <jj@test.net> wrote:
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it like
you said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:
> It's taking forever to upload 400,000 records to the database through
> access/odbc, and I've tried phpMyAdmin's interface but it seems totimeout > during import of a CSV file. Is there a better way to import 400,000+
> records?


If it is possible for you to get the data into sql command-form like this:

-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt


Jul 19 '05 #3

P: n/a
jj wrote:
It's taking forever to upload 400,000 records to the database through
access/odbc, and I've tried phpMyAdmin's interface but it seems to timeout
during import of a CSV file. Is there a better way to import 400,000+
records?


If it is possible for you to get the data into sql command-form like this:

-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt

Jul 19 '05 #4

P: n/a
jj
from Access via MyODBC for MySQL to my local MySQL server it took only a
minute or so. But through ODBC to a remote server, it takes hours then
finally dies. We have a high speed connection. Weird...
<us******@tampabay.rr.com> wrote in message
news:av********************************@4ax.com...
Don't listen to the previous guy.

Use LOAD DATA CONCURRENT INFILE

Loading of several megs of data will take only a few seconds.

On Fri, 19 Dec 2003 19:30:04 GMT, "jj" <jj@test.net> wrote:
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it likeyou said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:

> It's taking forever to upload 400,000 records to the database through
> access/odbc, and I've tried phpMyAdmin's interface but it seems to

timeout
> during import of a CSV file. Is there a better way to import 400,000+ > records?

If it is possible for you to get the data into sql command-form like this:
-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt

Jul 19 '05 #5

P: n/a
jj
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it like
you said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:
It's taking forever to upload 400,000 records to the database through
access/odbc, and I've tried phpMyAdmin's interface but it seems to timeout during import of a CSV file. Is there a better way to import 400,000+
records?


If it is possible for you to get the data into sql command-form like this:

-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt

Jul 19 '05 #6

P: n/a
Don't listen to the previous guy.

Use LOAD DATA CONCURRENT INFILE

Loading of several megs of data will take only a few seconds.

On Fri, 19 Dec 2003 19:30:04 GMT, "jj" <jj@test.net> wrote:
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it like
you said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:
> It's taking forever to upload 400,000 records to the database through
> access/odbc, and I've tried phpMyAdmin's interface but it seems totimeout > during import of a CSV file. Is there a better way to import 400,000+
> records?


If it is possible for you to get the data into sql command-form like this:

-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt


Jul 19 '05 #7

P: n/a
jj
from Access via MyODBC for MySQL to my local MySQL server it took only a
minute or so. But through ODBC to a remote server, it takes hours then
finally dies. We have a high speed connection. Weird...
<us******@tampabay.rr.com> wrote in message
news:av********************************@4ax.com...
Don't listen to the previous guy.

Use LOAD DATA CONCURRENT INFILE

Loading of several megs of data will take only a few seconds.

On Fri, 19 Dec 2003 19:30:04 GMT, "jj" <jj@test.net> wrote:
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it likeyou said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:

> It's taking forever to upload 400,000 records to the database through
> access/odbc, and I've tried phpMyAdmin's interface but it seems to

timeout
> during import of a CSV file. Is there a better way to import 400,000+ > records?

If it is possible for you to get the data into sql command-form like this:
-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt

Jul 19 '05 #8

This discussion thread is closed

Replies have been disabled for this discussion.