469,579 Members | 1,272 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,579 developers. It's quick & easy.

How to speed up the import of 400,000+ records

jj
It's taking forever to upload 400,000 records to the database through
access/odbc, and I've tried phpMyAdmin's interface but it seems to timeout
during import of a CSV file. Is there a better way to import 400,000+
records?

Jul 19 '05 #1
7 9061
jj
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it like
you said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:
It's taking forever to upload 400,000 records to the database through
access/odbc, and I've tried phpMyAdmin's interface but it seems to timeout during import of a CSV file. Is there a better way to import 400,000+
records?


If it is possible for you to get the data into sql command-form like this:

-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt

Jul 19 '05 #2
Don't listen to the previous guy.

Use LOAD DATA CONCURRENT INFILE

Loading of several megs of data will take only a few seconds.

On Fri, 19 Dec 2003 19:30:04 GMT, "jj" <jj@test.net> wrote:
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it like
you said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:
> It's taking forever to upload 400,000 records to the database through
> access/odbc, and I've tried phpMyAdmin's interface but it seems totimeout > during import of a CSV file. Is there a better way to import 400,000+
> records?


If it is possible for you to get the data into sql command-form like this:

-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt


Jul 19 '05 #3
jj wrote:
It's taking forever to upload 400,000 records to the database through
access/odbc, and I've tried phpMyAdmin's interface but it seems to timeout
during import of a CSV file. Is there a better way to import 400,000+
records?


If it is possible for you to get the data into sql command-form like this:

-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt

Jul 19 '05 #4
jj
from Access via MyODBC for MySQL to my local MySQL server it took only a
minute or so. But through ODBC to a remote server, it takes hours then
finally dies. We have a high speed connection. Weird...
<us******@tampabay.rr.com> wrote in message
news:av********************************@4ax.com...
Don't listen to the previous guy.

Use LOAD DATA CONCURRENT INFILE

Loading of several megs of data will take only a few seconds.

On Fri, 19 Dec 2003 19:30:04 GMT, "jj" <jj@test.net> wrote:
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it likeyou said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:

> It's taking forever to upload 400,000 records to the database through
> access/odbc, and I've tried phpMyAdmin's interface but it seems to

timeout
> during import of a CSV file. Is there a better way to import 400,000+ > records?

If it is possible for you to get the data into sql command-form like this:
-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt

Jul 19 '05 #5
jj
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it like
you said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:
It's taking forever to upload 400,000 records to the database through
access/odbc, and I've tried phpMyAdmin's interface but it seems to timeout during import of a CSV file. Is there a better way to import 400,000+
records?


If it is possible for you to get the data into sql command-form like this:

-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt

Jul 19 '05 #6
Don't listen to the previous guy.

Use LOAD DATA CONCURRENT INFILE

Loading of several megs of data will take only a few seconds.

On Fri, 19 Dec 2003 19:30:04 GMT, "jj" <jj@test.net> wrote:
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it like
you said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:
> It's taking forever to upload 400,000 records to the database through
> access/odbc, and I've tried phpMyAdmin's interface but it seems totimeout > during import of a CSV file. Is there a better way to import 400,000+
> records?


If it is possible for you to get the data into sql command-form like this:

-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt


Jul 19 '05 #7
jj
from Access via MyODBC for MySQL to my local MySQL server it took only a
minute or so. But through ODBC to a remote server, it takes hours then
finally dies. We have a high speed connection. Weird...
<us******@tampabay.rr.com> wrote in message
news:av********************************@4ax.com...
Don't listen to the previous guy.

Use LOAD DATA CONCURRENT INFILE

Loading of several megs of data will take only a few seconds.

On Fri, 19 Dec 2003 19:30:04 GMT, "jj" <jj@test.net> wrote:
Thanks that gave me an idea. I'm importing all the records to my local
MySQL database and will create an extract of that which will format it likeyou said. Then take that .sql file and import it onto the server. We'll
see...
"Aggro" <sp**********@yahoo.com> wrote in message
news:X9***************@read3.inet.fi...
jj wrote:

> It's taking forever to upload 400,000 records to the database through
> access/odbc, and I've tried phpMyAdmin's interface but it seems to

timeout
> during import of a CSV file. Is there a better way to import 400,000+ > records?

If it is possible for you to get the data into sql command-form like this:
-----filename.txt--------
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
insert into tablename(column1,column2) values(xx,yy);
-----filename.txt--------

You can insert it into database using mysql console like this:

c:\mysql\bin\mysql -u username databasename < filename.txt

Jul 19 '05 #8

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

14 posts views Thread by Bulba! | last post: by
17 posts views Thread by Shailesh Humbad | last post: by
reply views Thread by Creigh Shank | last post: by
3 posts views Thread by Reddy | last post: by
11 posts views Thread by Sezai YILMAZ | last post: by
9 posts views Thread by Paul | last post: by
4 posts views Thread by guiromero | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.