473,549 Members | 2,719 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Simple script to merge multiple text files

7 New Member
Hi,

I have just started using Python and I am slowly getting into it. I wanted to make a little script to merge all files in a directory into one.

All of these files will be text files.

I know I need to use os.path.walk() to walk through all the files, and if not os.path.isdir(f ilename) with open(filename,' r').read() to read all the files that are not directories.

Can someone give me a clue or a very simple example that I can work with please?

Thanks in advance.
Nov 1 '07 #1
6 30646
bvdet
2,851 Recognized Expert Moderator Specialist
Hi,

I have just started using Python and I am slowly getting into it. I wanted to make a little script to merge all files in a directory into one.

All of these files will be text files.

I know I need to use os.path.walk() to walk through all the files, and if not os.path.isdir(f ilename) with open(filename,' r').read() to read all the files that are not directories.

Can someone give me a clue or a very simple example that I can work with please?

Thanks in advance.
I think this code will do what you want:
Expand|Select|Wrap|Line Numbers
  1. import os
  2.  
  3. def dir_list(dir_name, subdir, *args):
  4.     '''Return a list of file names in directory 'dir_name'
  5.     If 'subdir' is True, recursively access subdirectories under 'dir_name'.
  6.     Additional arguments, if any, are file extensions to add to the list.
  7.     Example usage: fileList = dir_list(r'H:\TEMP', False, 'txt', 'py', 'dat', 'log', 'jpg')
  8.     '''
  9.     fileList = []
  10.     for file in os.listdir(dir_name):
  11.         dirfile = os.path.join(dir_name, file)
  12.         if os.path.isfile(dirfile):
  13.             if len(args) == 0:
  14.                 fileList.append(dirfile)
  15.             else:
  16.                 if os.path.splitext(dirfile)[1][1:] in args:
  17.                     fileList.append(dirfile)
  18.  
  19.         # recursively access file names in subdirectories
  20.         elif os.path.isdir(dirfile) and subdir:
  21.             # print "Accessing directory:", dirfile
  22.             fileList += dir_list(dirfile, subdir, *args)
  23.     return fileList
  24.  
  25. def combine_files(fileList, fn):
  26.     f = open(fn, 'w')
  27.     for file in fileList:
  28.         print 'Writing file %s' % file
  29.         f.write(open(file).read())
  30.     f.close()
  31.  
  32. if __name__ == '__main__':
  33.     search_dir = "C:/directory"
  34.     fn = "output_file.txt"
  35.     combine_files(dir_list(search_dir, False, 'txt'), fn)
Nov 1 '07 #2
borthouth
7 New Member
Thanks , you have helped a lot :)
Nov 1 '07 #3
ghostdog74
511 Recognized Expert Contributor
another way
Expand|Select|Wrap|Line Numbers
  1. import os,shutil
  2. f=open("/tmp/fileappend.txt","a")
  3. for r,d,fi in os.walk("/home/me"):
  4.     for files in fi:
  5.         if files.endswith(".txt"):                         
  6.             g=open(os.path.join(r,files))
  7.             shutil.copyfileobj(g,f)
  8.             g.close()
  9. f.close()
  10.  
Nov 2 '07 #4
borthouth
7 New Member
I think this code will do what you want:
Expand|Select|Wrap|Line Numbers
  1. import os
  2.  
  3. def dir_list(dir_name, subdir, *args):
  4.     '''Return a list of file names in directory 'dir_name'
  5.     If 'subdir' is True, recursively access subdirectories under 'dir_name'.
  6.     Additional arguments, if any, are file extensions to add to the list.
  7.     Example usage: fileList = dir_list(r'H:\TEMP', False, 'txt', 'py', 'dat', 'log', 'jpg')
  8.     '''
  9.     fileList = []
  10.     for file in os.listdir(dir_name):
  11.         dirfile = os.path.join(dir_name, file)
  12.         if os.path.isfile(dirfile):
  13.             if len(args) == 0:
  14.                 fileList.append(dirfile)
  15.             else:
  16.                 if os.path.splitext(dirfile)[1][1:] in args:
  17.                     fileList.append(dirfile)
  18.  
  19.         # recursively access file names in subdirectories
  20.         elif os.path.isdir(dirfile) and subdir:
  21.             # print "Accessing directory:", dirfile
  22.             fileList += dir_list(dirfile, subdir, *args)
  23.     return fileList
  24.  
  25. def combine_files(fileList, fn):
  26.     f = open(fn, 'w')
  27.     for file in fileList:
  28.         print 'Writing file %s' % file
  29.         f.write(open(file).read())
  30.     f.close()
  31.  
  32. if __name__ == '__main__':
  33.     search_dir = "C:/directory"
  34.     fn = "output_file.txt"
  35.     combine_files(dir_list(search_dir, False, 'txt'), fn)
This seemed to work for a while for me now it comes up with "NameError: name 'False' is not defined"
Nov 2 '07 #5
bvdet
2,851 Recognized Expert Moderator Specialist
This seemed to work for a while for me now it comes up with "NameError: name 'False' is not defined"
Expand|Select|Wrap|Line Numbers
  1. >>> bool(0)
  2. False
  3. >>> bool(1)
  4. True
  5. >>> 
'0' and '1' can be substituted for 'False' and 'True' respectively. The question is what happened to 'False' on your system?
Nov 2 '07 #6
borthouth
7 New Member
Expand|Select|Wrap|Line Numbers
  1. >>> bool(0)
  2. False
  3. >>> bool(1)
  4. True
  5. >>> 
'0' and '1' can be substituted for 'False' and 'True' respectively. The question is what happened to 'False' on your system?
Did a quick reset on my machine and is working now. very strange.
Nov 9 '07 #7

Sign in to post your reply or Sign up for a free account.

Similar topics

0
3664
by: Guillaume Lahitette | last post by:
Hello everyone, I am looking for a lightweight Java Applet to diff and merge two text files. The functionality would be a subset of what WinMerge (http://winmerge.sourceforge.net) offers: - diff 2 text files - merge (in both directions) - color coding to highlight differences I found some Java diff tools but they were all command line....
1
6568
by: Charlie | last post by:
Hello, I have data in an Access table that I would like to export to multiple HTML tables. I would like to split the data in the Access table (about 92,000 records) into multiple HTML tables/files to reduce download time and bandwidth usage on my web server. That way, the user can select a particular page to download instead of...
2
1730
by: Chris Murphy via DotNetMonster.com | last post by:
Hey all, just wondering if anyone can point me in the right direction. I'm developing a solution that allows a user to store multiple text-based content (like code snippets, notes, documents etc.) in one master document. I'm not exactly sure which is the most efficient method of going about this. Should I: a. Use a container file like a CAB...
5
3420
by: Buddhist[CHinA] | last post by:
The text files are not only the .txt files, but also all ascii files. Thx.
9
8837
by: malla | last post by:
Hi, I am trying to open a bunch of files and get data from them into one single file. I am sure this process can be automated if I name the files that I want to open in a regular pattern (say 1.txt, 2.txt, 3.txt, ...) Has anyone done sth like this before? Is there a way to achieve this easily? Please share.
2
2702
by: ManningFan | last post by:
I'm running through a database in a loop, and each time the loop finishes I would like to export the values of some variables to new text files. In reality it's more complex than this, but this is the simplest way to state it. Is there a way to write to text files and save them as different names (presumably one of the variables dot "TXT")...
3
5673
emaghero
by: emaghero | last post by:
Hello all, I want to open multiple txt files with similar names in C++ I have attempted this with the following code //Create as many txt files as there are valid propagation constants //Propagation constants are elements of the vector beta for(int j=0;j<(int)(number_of_nodes);j++){ if(*(beta+j)>substrate*k && *(beta+i)<core*k){...
2
4182
by: as001 | last post by:
Hi, I'm writing a windows application in C# using VS 2003. I got stuck where it has to write multiple output text files. Here's my piece of code: for loop { Random r = new Random(); //use millisecond and random to avoid similar filename
5
14625
by: maral | last post by:
Hi every one, this is my first post here! I'm using GATE toolkit for information retrieval and text analysis, but i really need java for some parts. I have managed to find a specific word in one text file, but i need to look for a specific word, in multiple text files. all of the files are ".txt" and they are in the same folder but each of...
0
7520
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7957
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
0
6043
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5368
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5088
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3500
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3481
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1059
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
763
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.