473,231 Members | 1,840 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,231 software developers and data experts.

linecache and glob

hi everyone happy new year!
im a newbie to python
i have a question
by using linecache and glob
how do i read a specific line from a file in a batch and then insert
it into database?

because it doesn't work! i can't use glob wildcard with linecache
>>import linecache
linecache.getline(glob.glob('/etc/*', 4)
doens't work

is there any better methods??? thank you very much in advance

jo3c
Jan 4 '08 #1
5 2378
Hello,

Welcome to Python!

glob returns a list of filenames, but getline is made to work on just
one filename.
So you'll need to iterate over the list returned by glob.
>>import linecache, glob
for filename in glob.glob('/etc/*'):
print linecache.getline(filename, 4)
Maybe you could explain more about what you are trying to do and we
could help more?

Hope this helps,

Jeremy

On Jan 3, 10:02 pm, jo3c <JO3chi...@gmail.comwrote:
hi everyone happy new year!
im a newbie to python
i have a question
by using linecache and glob
how do i read a specific line from a file in a batch and then insert
it into database?

because it doesn't work! i can't use glob wildcard with linecache
>import linecache
linecache.getline(glob.glob('/etc/*', 4)

doens't work

is there any better methods??? thank you very much in advance

jo3c
Jan 4 '08 #2
i have a 2000 files with header and data
i need to get the date information from the header
then insert it into my database
i am doing it in batch so i use glob.glob('/mydata/*/*/*.txt')
to get the date on line 4 in the txt file i use
linecache.getline('/mydata/myfile.txt/, 4)

but if i use
linecache.getline('glob.glob('/mydata/*/*/*.txt', 4) won't work

i am running out of ideas

thanks in advance for any help

jo3c
Jan 4 '08 #3
import linecache
import glob

# reading from one file
print linecache.getline('notes/python.txt',4)
'http://www.python.org/doc/current/lib/\n'

# reading from many files
for filename in glob.glob('/etc/*'):
print linecache.getline(filename,4)


jo3c wrote:
hi everyone happy new year!
im a newbie to python
i have a question
by using linecache and glob
how do i read a specific line from a file in a batch and then insert
it into database?

because it doesn't work! i can't use glob wildcard with linecache

>>>import linecache
linecache.getline(glob.glob('/etc/*', 4)

doens't work

is there any better methods??? thank you very much in advance

jo3c

--
Shane Geiger
IT Director
National Council on Economic Education
sg*****@ncee.net | 402-438-8958 | http://www.ncee.net

Leading the Campaign for Economic and Financial Literacy

Jan 4 '08 #4
jo3c wrote:
i have a 2000 files with header and data
i need to get the date information from the header
then insert it into my database
i am doing it in batch so i use glob.glob('/mydata/*/*/*.txt')
to get the date on line 4 in the txt file i use
linecache.getline('/mydata/myfile.txt/, 4)

but if i use
linecache.getline('glob.glob('/mydata/*/*/*.txt', 4) won't work
glob.glob returns a list of filenames, so you need to call getline once
for each file in the list.

but using linecache is absolutely the wrong tool for this; it's designed
for *repeated* access to arbitrary lines in a file, so it keeps all the
data in memory. that is, all the lines, for all 2000 files.

if the files are small, and you want to keep the code short, it's easier
to just grab the file's content and using indexing on the resulting list:

for filename in glob.glob('/mydata/*/*/*.txt'):
line = list(open(filename))[4-1]
... do something with line ...

(note that line numbers usually start with 1, but Python's list indexing
starts at 0).

if the files might be large, use something like this instead:

for filename in glob.glob('/mydata/*/*/*.txt'):
f = open(filename)
# skip first three lines
f.readline(); f.readline(); f.readline()
# grab the line we want
line = f.readline()
... do something with line ...

</F>

Jan 4 '08 #5
On Jan 4, 5:25 pm, Fredrik Lundh <fred...@pythonware.comwrote:
jo3c wrote:
i have a 2000 files with header and data
i need to get the date information from the header
then insert it into my database
i am doing it in batch so i use glob.glob('/mydata/*/*/*.txt')
to get the date on line 4 in the txt file i use
linecache.getline('/mydata/myfile.txt/, 4)
but if i use
linecache.getline('glob.glob('/mydata/*/*/*.txt', 4) won't work

glob.glob returns a list of filenames, so you need to call getline once
for each file in the list.

but using linecache is absolutely the wrong tool for this; it's designed
for *repeated* access to arbitrary lines in a file, so it keeps all the
data in memory. that is, all the lines, for all 2000 files.

if the files are small, and you want to keep the code short, it's easier
to just grab the file's content and using indexing on the resulting list:

for filename in glob.glob('/mydata/*/*/*.txt'):
line = list(open(filename))[4-1]
... do something with line ...

(note that line numbers usually start with 1, but Python's list indexing
starts at 0).

if the files might be large, use something like this instead:

for filename in glob.glob('/mydata/*/*/*.txt'):
f = open(filename)
# skip first three lines
f.readline(); f.readline(); f.readline()
# grab the line we want
line = f.readline()
... do something with line ...

</F>
thank you guys, i did hit a wall using linecache, due to large file
loading into memory.. i think this last solution works well for me
thanks
Jan 8 '08 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

15
by: Georgy Pruss | last post by:
On Windows XP glob.glob doesn't work properly for files without extensions. E.g. C:\Temp contains 4 files: 2 with extensions, 2 without. C:\Temp>dir /b * aaaaa.aaa bbbbb.bbb ccccc ddddd ...
5
by: Elbert Lev | last post by:
#Here is the script #Python 2.3 on W2K import glob name = glob.glob(u"./*.mp3") print type(name) name = glob.glob(u"*.mp3") print type(name)
4
by: Python Dunce | last post by:
I've run into an issue with glob and matching filenames with brackets '' in them. The problem comes when I'm using part of such a filename as the path I'm passing to glob. Here's a trimmed down...
2
by: Zain Homer | last post by:
Someone please let me know if I'm sending this to the wrong email alias... I'm wondering why we can't use the glob module to glob with curly brackets like we can from the command line (at least...
6
by: Hitesh | last post by:
import string import os f = open ("c:\\servername.txt", 'r') linelist = f.read() lineLog = string.split(linelist, '\n') lineLog = lineLog #print lineLog for l in lineLog:
8
by: mark.bergman | last post by:
I am porting from Digital Unix to Linux (RHEL 4), and am seeing a difference in the return value of glob(). Given a non-existant directory "/tmp/a", and the following line of code: result =...
3
by: billiejoex | last post by:
os.listdir(path) return a list of file names independently if the path- argument used is absolute or relative: '/home/user' glob.glob, instead, return file names only if given path is...
2
by: Ross Hetherington | last post by:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Say I've got a file called 'testfile' that contains the following three lines: foo bar baz
1
by: crybaby | last post by:
when I do this in my python code and run it in windows xp, it creates ctemp/..../.../.../../ so on and creates file t. Not file starting with the name complist and ending with .txt...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.