473,287 Members | 1,613 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

RE: Fixed-length text file to database script

#your thought is right.
================================================== =====
def sizes2fields(sizes):
d = []
begin = 0
for i in sizes:
if begin:
end = begin + i
else: end = i
d.append((begin, end))
begin += i
return tuple(d)

def slicestring(s, fields):
d = []
for i in fields:
d.append(s[i[0]:i[1]])
return tuple(d)

sizes = [16,4,8,8,8]
s = '1234567890123456789012345678901234567890123456789 01234567890'
print slicestring(s, sizes2fields(sizes))
================================================== ========
prints out:
('1234567890123456', '7890', '12345678', '90123456', '78901234')

hope it helps.
thanks Edwin

-----Original Message-----
From: py************************************************ **@python.org
[mailto:py***************************************** *********@python.org]
On Behalf Of Eric Wertman
Sent: Thursday, August 14, 2008 1:59 PM
To: py*********@python.org
Subject: Re: Fixed-length text file to database script
I have a machine (PLC) that is dumping its test results into a fixed-
length text file.
While it has nothing to do with python, I found that creating a MySQL
table with the proper fixed length char() fields and using 'load data
infile' was the easiest way to deal with that sort of scenario. The
python script is the secondary part, that handles the normalization
and proper typing of the first table to the second, permanent storage
area. But in this case, the more advanced bits are the database and
SQL details, and python is just a very convenient way to build the SQL
statements and execute them.

I'm really not sure what the best way to deal with fixed length data
is in python. I might define a list with the field lengths and use a
string slicing to get the items.. as a first thought:

myfile = '/somewhere/somefile.txt'
sizes = [16,4,8,8,8]

fd = open(myfile,r)

for line in fd.readlines() :

idx1 = 0
for l in sizes :
--
http://mail.python.org/mailman/listinfo/python-list
The information contained in this message and any attachment may be
proprietary, confidential, and privileged or subject to the work
product doctrine and thus protected from disclosure. If the reader
of this message is not the intended recipient, or an employee or
agent responsible for delivering this message to the intended
recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please notify me
immediately by replying to this message and deleting it and all
copies and backups thereof. Thank you.
Aug 14 '08 #1
2 3123
On Aug 15, 4:55 am, Edwin.Mad...@VerizonWireless.com wrote:
#your thought is right.
================================================== =====
def sizes2fields(sizes):
d = []
begin = 0
for i in sizes:
if begin:
end = begin + i
else: end = i
d.append((begin, end))
begin += i
return tuple(d)
Those who are not paid by the keystroke and/or prefer to expend
keystrokes on meaningful names might like an alternative like this:
def sizes2offsets(sizes):
offsets = []
begin = 0
for size in sizes:
end = begin + size
offsets.append((begin, end))
begin = end
return offsets
Aug 15 '08 #2
Lie
On Aug 15, 7:12*am, John Machin <sjmac...@lexicon.netwrote:
On Aug 15, 4:55 am, Edwin.Mad...@VerizonWireless.com wrote:
#your thought is right.
================================================== =====
def sizes2fields(sizes):
* *d = []
* *begin = 0
* *for i in sizes:
* * * if begin:
* * * * *end = begin + i
* * * else: end = i
* * * d.append((begin, end))
* * * begin += i
* *return tuple(d)

Those who are not paid by the keystroke and/or prefer to expend
keystrokes on meaningful names might like an alternative like this:
def sizes2offsets(sizes):
* offsets = []
* begin = 0
* for size in sizes:
* * end = begin + size
* * offsets.append((begin, end))
* * begin = end
* return offsets
This is even shorter: (and IMHO, clearer)

def split(s, fields):
ret = []
for field in fields:
s, d = s[field:], s[:field]
ret.append(d)
return ret

sizes = [16, 4, 8, 8, 8]
s = '1234567890123456789012345678901234567890123456789 01234567890'
print split(s, sizes)

alternatively, if what you've got is the partition position instead of
field width:

def split(s, sizes):
ret = []
start = sizes[0]
for end in sizes[1:]:
r, start = s[start: end], end
ret.append(r)
return ret

sizes = [0, 16, 20, 28, 36, 44]
s = '1234567890123456789012345678901234567890123456789 01234567890'
print split(s, sizes)

Michael Stroder:
If the input data has to be pre-processed before storing it into the
database a Python script would be needed.
But not for converting the fixed length string, you could just have
SQL process the fixed length string then retrieve it back as separate
fields.
Aug 15 '08 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

9
by: Paul Trautwein | last post by:
I'm trying to get an image to float in a window despite scrolling. I've gotten it to work on my Mac using IE 5.2, Netscape, and Safari, but it goes wonky when I test it on a PC. (testing with IE...
6
by: Mason A. Clark | last post by:
Masters: On two or three-column layouts, one column often has a list of links. Scrolling the page hides them. I'm aware there's supposed to be the ability to fix the column (frame-like). I...
5
by: Ted Mayett | last post by:
I've read a lot through the google archives, but I cannot seem to find an example... Has someone successfully made css code that allows for position: fixed to work in IE 6.0.2800? What is the...
2
by: hq105862 | last post by:
Hi, Is it possible to simulate frames behaviour with CSS? Currently, I've used CSS to create the visual look of my old framed site. For this I've used styles that place the images at absolute...
2
by: Jonathan Carmichael | last post by:
I'm trying to create a fixed, no-repeat, centered background image using an external css. Everything works great until I add background-attachment: fixed then it just doesn't show the image...
9
by: pout | last post by:
What are the purposes of fixed-point? When should it be used? I read: #define Int2Fixed(x) (((long)(short)x) << 16) and the fixed-point in 16.16 format. Does the 16 in the MACRO refer to...
3
by: Chris Dunaway | last post by:
I was using .Net Reflector to look at some methods in the String class and found this one: Friend Sub AppendInPlace(ByVal value As Char*, ByVal count As Integer, ByVal currentLength As Integer)...
4
by: Otie | last post by:
Hello, I am using the MSFlexGrd Control in VB5. I have 1 fixed row and one fixed column. I am trying to do a sort when the user clicks a column in the FIXED ROW. But when I capture the row...
2
by: Eric Lindsay | last post by:
Googling suggests that IE7 may support position: fixed; I think this might be handy for some pages I want to do. Does anyone have any comments about whether fixed should be considered for use on...
8
by: ms news group | last post by:
What happens if exception is thown within a fixed block? Will the pinned memory buffer get unpinned? and if the pinning pointer points to a managed memery buffer allocated within the throwing...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.