Hi,
I'm not a programmer. I start working as text miner and as a first task
I have given 1000 dirty files that needs to be cleaned before
classification tasks. I have been told python is the best tool for this
job.
Each file's structure as below:
Comments: This is article 1965 obtained from the website
Title: Banana Report #65, September 2003
Author: dylab
Date: 1st September 2003
Section: pulse
In the past month:
A mass hit North America, cutting electricity to 50 million people
across the North east
I'm expected execute the python script so the file suppose to look like
this:
pulse, In, the, past, month, A, mass, hit, North, America, cutting,
electricity, to, 50, million, people, across, the, North east, dylab
Could you please point me to right direction here. Or provide some
example code. In the mean time I'll be searching myself. I know you
guys hate novice people like me but I would appreciated if you could
provide little help here.
Thanks & regards,
Sez 2 2178
Sez sez: Each file's structure as below: Comments: This is article 1965 obtained from the website Title: Banana Report #65, September 2003 Author: dylab Date: 1st September 2003 Section: pulse
In the past month: A mass hit North America, cutting electricity to 50 million people across the North east
I'm expected execute the python script so the file suppose to look like this:
pulse, In, the, past, month, A, mass, hit, North, America, cutting, electricity, to, 50, million, people, across, the, North east, dylab
You'll need either more examples or a more detailed description. The
above could be interpreted as something like "put the pulse section
first, then exactly 19 words from the following text, removing
punctuation and line breaks, and taking the last two words together as
one, then add the 'author' field, and write them all out together with a
field separator of ', ' (comma plus space)".
On the other hand, it could be interpreted a large number of other ways,
and since none of us have any idea what you are trying to do with the
results, we can't use our own intuition or experience to help.
I also personally find it hard to respond to questions like this with
real code when there are things about the task which I find very
surprising. For example, you're throwing away the date information
entirely, along with the comments and title. Is that really intended?
And are the author and section fields always exactly one word, with no
punctuation? (What would happen if an author's name was "Hansen,
Peter"? How would you format that in the output without getting the
first name confused with the next field?)
Could you please point me to right direction here. Or provide some example code. In the mean time I'll be searching myself. I know you guys hate novice people like me but I would appreciated if you could provide little help here.
We don't "hate" novice people by any means... I suspect you are either
trying to be self-deprecating or maybe you just haven't read this
newsgroup for long. c.l.p actually *loves* novices; it just doesn't
prefer questions that aren't very clear. Keep trying (and improving!)
and you'll definitely get the help you need.
And your comment about Python being the best language for this is pretty
close to the mark... but there are certainly a variety of ways to go
about the task and the best might depend on a lot of unanswered questions.
-Peter
On 8 May 2005 21:55:04 -0700, Sez <se*******@yaho o.com.au> wrote: Could you please point me to right direction here. Or provide some example code. In the mean time I'll be searching myself. I know you guys hate novice people like me but I would appreciated if you could provide little help here.
Oh, we don't hate novices here, not at all. On the other hand, we
aren't going to write your script for you. ;-) Why not take a look at
the Python beginners guide (at
<http://www.python.org/moin/BeginnersGuide> ), and come back to us when
you have a specific problem.
--
Cheers,
Simon B, si***@brunningo nline.net, http://www.brunningonline.net/simon/blog/ This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Shalen chhabra |
last post by:
Hey,
Can anyone give me a snippet for running a python program over all the files
in the directory.
For ex: I have ten files in a directory and I want to run a python program
against all of these files, I wish to do the same using another python code
instead of running each of these files one by one, which would be cumbersome
giving the argv of each file every single time.
This can be easily done using a shell script but I just...
|
by: Batista, Facundo |
last post by:
People:
I'm trying to convert my father from using COBOL to Python, :)
One difficult thing we stuck into is how to read, from python, files written
with COBOL.
Do you know a module that allows me to do that?
It should avoid us the work to write a COBOL program that open the COBOL
|
by: Tim Mierzejewski |
last post by:
How do I clear the text from my screen, other than a bunch of \n's or
endl's?
Tim M.
|
by: Sylvain Thenault |
last post by:
Hi there !
I've some questions regarding pth files (which btw are undocumented in the
python reference, is this intentional ?)
I thought that I could use a .pth file to be able to import zope products
from both INSTANCE_HOME/Products and ZOPE_HOME/lib/python/Products from
outside zope:
|
by: mp |
last post by:
i have a python program which attempts to call 'cls' but fails:
sh: line 1: cls: command not found
i tried creating an alias from cls to clear in .profile, .cshrc, and
/etc/profile, but none of these options seem to work.
my conclusion is that a python program that is executing does not use
the shell (because it does not recognize shell aliases). is this
correct?
| |
by: Ville Vainio |
last post by:
I tried to clear a list today (which I do rather rarely, considering
that just doing l = works most of the time) and was shocked, SHOCKED
to notice that there is no clear() method. Dicts have it, sets have it,
why do lists have to be second class citizens?
|
by: metaperl |
last post by:
A Comparison of Python Class Objects and Init Files for Program
Configuration
=============================================================================
Terrence Brannon
bauhaus@metaperl.com
http://www.livingcosmos.org/Members/sundevil/python/articles/a-comparison-of-python-class-objects-and-init-files-for-program-configuration/view
|
by: Marko.Cain.23 |
last post by:
Hi,
I create a dictionary like this
myDict = {}
and I add entry like this:
myDict = 1
but how can I empty the whole dictionary?
Thank you.
|
by: Albert-jan Roskam |
last post by:
Hi John,
Thanks! Using a higher xlrd version did the trick! Regarding your other remarks:
-yep, input files with multiple sheets don't work yet. I kinda repressed that ;-) Spss outputs only single-sheet xls files, but I agree it's nicer if the programs works in other cases too.
-and no, I don't intend to use data fields. Wouldn't it be easier to convert those to string values if I ever came across them?
Thanks again!
Albert-Jan
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |