472,958 Members | 2,642 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,958 software developers and data experts.

parsing flat file to mssql (odbc)

ok, fairly new to python, relatively familiar w/ ms-sql. here's my
issue:

my .py script parses the contents of a (tab delim.) flat file, then
attempts to insert the information into the db. I've been fighting
this for a day or so, and i'm stuck on the db insertion...

basically the script uses .readlines() to capture the flat file
contents, and stick the *record* into a container list...

something to the effect of:
for l in lines:
fields = string.split(l,'\t')
dic.append(fields)

then, I just want to loop over that container list and do a simple
INSERT statement. I've tried using the .executemany() method to no
avail. .execute(sql, tuple) seems to be working better.

sql statement (lots of columns):
---------------------------------
INSERT INTO tbl_pyDev
(COURSE_SECTIONS_0,Term_1,Synonym_2,Section_Name_3 ,Location_4,Bldg_5,Room_6,
Days_7,Start_Time_8,End_Time_9,Start_Date_10,End_D ate_11,Add_Start_Date_12,
Drop_Start_Date_13,Add_End_Date_14,Faculty_15,Shor t_Title_16,Prerequisite_17,
Required_18,Coreq_Noncourses_19,Course_20,Cred_Typ e_21,SEC_CRS_DESC_22,
Long_Title_23,Depts_24,Fee_25,Meeting_Days_26,Prin ted_Comments_27,Subject_28,
Supplies_29,Transfer_Status_30,Course_Cost_31,Stat us_32,Capacity_33,
COURSE_SECTIONS_34,Corequisite_Sections_35,Section _36,Min_Cred_37,
Instr_Methods_38,SEC_FACULTY_FIRST_NAME_39,SEC_FAC ULTY_LAST_NAME_40,
Refund_41,CoReq_Name_42,Drop_End_43) values (?,?,?,?,?,?,?,?,?,?,?,?,
?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?, ?,?,?,?,?,?,?)

ugly eh?
well, here's an example of what I'm trying to feed this beast:
(the first row of data is col. headers, and thy insert fine. This is
actually the chunk of data the script is choking on (dic[1]))
---------------------------------------------------------------
('2475', '03/FA', '25000', 'AAA-010-AS01', 'AS', 'AS', '116', 'TTH',
'05:00PM', '06:50PM', '09/09/03', '10/02/03', '09/09/03', '09/09/03',
'09/26/03', 'Rameil, Lesley J', 'Aca Achieve Pre-College', '', '', '',
'1139', 'UG', "Meets the requirements of the Comprehensive Student
AssessmentSystem and the Secretary's Commission on Achieving Necessary
Sklls, as well as work and postsecondary enrollment skills. Enabes
the student to review and improve in reading, writing, matheatics,
science, and social studies in preparation for the GED tst.",
'Academic Achievement in Pre-College', 'AAA', '', 'T', '', 'AAA', '',
'NT', '', 'A', '35', '2475', '', 'AS01', '1.00', 'LEC', 'Lesley',
'Rameil', '09/12/03', '', '09/26/03')

I'm executing in this sort of fashion:
--------------------------------------
for data in dic:
cursor.execute(insertSQL, tuple(data))

Error:
---------------------------------------
Traceback (most recent call last):
File "C:\Python22\parse_flat.py", line 101, in ?
cursor.execute(insertSQL, tuple(data))
dbi.internal-error: [Microsoft][ODBC SQL Server Driver][SQL
Server]Location: record.cpp:2253
Expression: m_futureSize == 0 || rec.Size () == m_futureSize
SPID: 58
Process ID: 1136 in EXEC

so, if the column headers make into the db, it has to be the
formatting of the following data?!?! single quotes?!?! i'm fried on
this...if anyone has any input I would be very grateful.

thanks - d
Jul 18 '05 #1
1 2624
>
Strange error message you have there...

Have you tried using mxODBC at this ? I'd bet you get much better
results.

--
Marc-Andre Lemburg
eGenix.com

Mark,
thanks for your suggestion, it seems to be working w/ mxODBC.
Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
by: mj | last post by:
Hello, thanks for the help. I am running a WinXP Pro w/ SP2 (my home computer, with ZoneAlarm firewall) Apache 2.0.52 MySQL 4.1.7 PHP 5.1.0-dev I have developed a PHP/MySQL web app that...
2
by: Josh Close | last post by:
Is there a python module that uses the mssql client tools to connect to mssql? When you install php on windows and you want mssql connectivity, you need the client tools and it uses that api to...
3
by: francisl | last post by:
We have to build some script were I work to make a dynamic server inventory. But, the project team, a windows crew, start it all in vbscript and on mssql. Note, due to political reason, we can...
12
by: Grahammer | last post by:
For some reason I am getting an error when trying to open a recordset on an Access database on my Win2K3 machine from my INDEX.ASP page, but the same code accesses the database fine when coming...
0
by: mwazir | last post by:
Dear all, I have a requirement while parsing a delimited flat file and I was wondering if there is a simpler solution that what I have in mind. Basically I need to pull out a line from the flat...
14
by: Kukurydz | last post by:
I've got such problem: My database is stored on MSSQL Server. I have to write reports for it in MSAccess. I've got a problem with creating a query which will select records from MSSQL table with...
2
by: Neural | last post by:
Hi, I was wondering if anybody knew of any other ways of efficiently parsing a flat file into SQL Server 2000 using C#. The flat files are tab delimited. And the general file size is around 1 GB...
4
by: kenneth.osenbroch | last post by:
Hi, I have a MSSQL Server communicating with an Oracle database through a MSSQL linked server using a MS ODBC connection. If I query the Oracle database through the Oracle ODBC 32Bit Test, the...
0
by: Gosth in the shell | last post by:
Hi there, i need some help I got a software to backend is MSSQL 2005, but a provider software requires an UPDATE on his table with MySQL 5.0.56a backend, so i made the next: download and...
2
by: DJRhino | last post by:
Was curious if anyone else was having this same issue or not.... I was just Up/Down graded to windows 11 and now my access combo boxes are not acting right. With win 10 I could start typing...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 4 Oct 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
0
by: Aliciasmith | last post by:
In an age dominated by smartphones, having a mobile app for your business is no longer an option; it's a necessity. Whether you're a startup or an established enterprise, finding the right mobile app...
2
by: giovanniandrean | last post by:
The energy model is structured as follows and uses excel sheets to give input data: 1-Utility.py contains all the functions needed to calculate the variables and other minor things (mentions...
4
NeoPa
by: NeoPa | last post by:
Hello everyone. I find myself stuck trying to find the VBA way to get Access to create a PDF of the currently-selected (and open) object (Form or Report). I know it can be done by selecting :...
3
NeoPa
by: NeoPa | last post by:
Introduction For this article I'll be using a very simple database which has Form (clsForm) & Report (clsReport) classes that simply handle making the calling Form invisible until the Form, or all...
1
by: Teri B | last post by:
Hi, I have created a sub-form Roles. In my course form the user selects the roles assigned to the course. 0ne-to-many. One course many roles. Then I created a report based on the Course form and...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 1 Nov 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM) Please note that the UK and Europe revert to winter time on...
0
isladogs
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.