473,493 Members | 2,245 Online
Bytes | Software Development & Data Engineering Community
Create Post

Home Posts Topics Members FAQ

Python Data Utils

In an effort to experiment with open source, I put a couple of my
utility files up <a href="http://github.com/jessald/python_data_utils/
tree/master">here</a>. What do you think?
Apr 6 '08 #1
6 1693
En Sun, 06 Apr 2008 01:43:29 -0300, Jesse Aldridge
<Je***********@gmail.comescribió:
In an effort to experiment with open source, I put a couple of my
utility files up <a href="http://github.com/jessald/python_data_utils/
tree/master">here</a>. What do you think?
Some names are a bit obscure - "universify"?
Docstrings would help too, and blank lines, and in general following PEP8
style guide.
find_string is a much slower version of the find method of string objects,
same for find_string_last, contains and others.
And I don't see what you gain from things like:
def count( s, sub ):
return s.count( sub )
it's slower and harder to read (because one has to *know* what S.count
does).
Other functions may be useful but without even a docstring it's hard to
tell what they do.
delete_string, as a function, looks like it should delete some string, not
return a character; I'd use a string constant DELETE_CHAR, or just DEL,
it's name in ASCII.

In general, None should be compared using `is` instead of `==`, and
instead of `type(x) is type(0)` or `type(x) == type(0)` I'd use
`isinstance(x, int)` (unless you use Python 2.1 or older, int, float, str,
list... are types themselves)

Files.py is similar - a lot of more or less common things with a different
name, and a few wheels reinvented :)

Don't feel bad, but I would not use those modules because there is no net
gain, and even a loss in legibility. If you develop your code alone,
that's fine, you know what you wrote and can use it whenever you please.
But for others to use it, it means that they have to learn new ways to say
the same old thing.

--
Gabriel Genellina

Apr 6 '08 #2
On Sun, Apr 6, 2008 at 7:43 AM, Jesse Aldridge <Je***********@gmail.comwrote:
In an effort to experiment with open source, I put a couple of my
utility files up <a href="http://github.com/jessald/python_data_utils/
tree/master">here</a>. What do you think?
Would you search for, install, learn and use these modules if *someone
else* created them?

--
kv
Apr 6 '08 #3
Thanks for the detailed feedback. I made a lot of modifications based
on your advice. Mind taking another look?
Some names are a bit obscure - "universify"?
Docstrings would help too, and blank lines
I changed the name of universify and added a docstrings to every
function.
...PEP8
I made a few changes in this direction, feel free to take it the rest
of the way ;)
find_string is a much slower version of the find method of string objects,*
Got rid of find_string, and contains. What are the others?
And I don't see what you gain from things like:
def count( s, sub ):
* * *return s.count( sub )
Yeah, got rid of that stuff too. I ported these files from Java a
while ago, so there was a bit of junk like this lying around.
delete_string, as a function, looks like it should delete some string, not*
return a character; I'd use a string constant DELETE_CHAR, or just DEL, *
it's name in ASCII.
Got rid of that too :)
In general, None should be compared using `is` instead of `==`, and *
instead of `type(x) is type(0)` or `type(x) == type(0)` I'd use *
`isinstance(x, int)` (unless you use Python 2.1 or older, int, float, str,*
list... are types themselves)
Changed.

So, yeah, hopefully things are better now.

Soon developers will flock from all over the world to build this into
the greatest data manipulation library the world has ever seen! ...or
not...

I'm tired. Making code for other people is too much work :)
Apr 6 '08 #4
On Apr 6, 6:14*am, "Konstantin Veretennicov" <kveretenni...@gmail.com>
wrote:
On Sun, Apr 6, 2008 at 7:43 AM, Jesse Aldridge <JesseAldri...@gmail.comwrote:
In an effort to experiment with open source, I put a couple of my
*utility files up <a href="http://github.com/jessald/python_data_utils/
*tree/master">here</a>. *What do you think?

Would you search for, install, learn and use these modules if *someone
else* created them?

--
kv
Yes, I would. I searched a bit for a library that offered similar
functionality. I didn't find anything. Maybe I'm just looking in the
wrong place. Any suggestions?
Apr 6 '08 #5
Docstrings go *after* the def statement.

Fixed.
changing "( " to "(" and " )" to ")".
Changed.
I attempted to take out everything that could be trivially implemented
with the standard library.
This has left me with... 4 functions in S.py. 1 one of them is used
internally, and the others aren't terribly awesome :\ But I think the
ones that remain are at least a bit useful :)
The penny drops :-)
yeah, yeah
Not in all places ... look at the ends_with function. BTW, this should
be named something like "fuzzy_ends_with".
fixed
fuzzy_match(None, None) should return False.
changed
2. make_fuzzy function: first two statements should read "s =
s.replace(.....)" instead of "s.replace(.....)".
fixed
3. Fuzzy matching functions are specialised to an application; I can't
imagine that anyone would be particularly interested in those that you
provide.
I think it's useful in many cases. I use it all the time. It helps
guard against annoying input errors.
A basic string normalisation-before-comparison function would
usefully include replacing multiple internal whitespace characters by
a single space.
I added this functionality.

5. Casual inspection of your indentation function gave the impression
that it was stuffed
Fixed

Thanks for the feedback.
Apr 7 '08 #6
On Apr 7, 4:22*pm, Jesse Aldridge <JesseAldri...@gmail.comwrote:
>
changing "( " to "(" and " )" to ")".

Changed.
But then you introduced more.
>
I attempted to take out everything that could be trivially implemented
with the standard library.
This has left me with... 4 functions in S.py. *1 one of them is used
internally, and the others aren't terribly awesome :\ *But I think the
ones that remain are at least a bit useful :)
If you want to look at stuff that can't be implemented trivially using
str/unicode methods, and is more than a bit useful, google for
mxTextTools.
>
A basic string normalisation-before-comparison function would
usefully include replacing multiple internal whitespace characters by
a single space.

I added this functionality.
Not quite. I said "whitespace", not "space".

The following is the standard Python idiom for removing leading and
trailing whitespace and replacing one or more whitespace characters
with a single space:

def normalise_whitespace(s):
return ' '.join(s.split())

If your data is obtained by web scraping, you may find some people use
'\xA0' aka NBSP to pad out fields. The above code will get rid of
these if s is unicode; if s is str, you need to chuck
a .replace('\xA0', ' ') in there somewhere.

HTH,
John

Apr 7 '08 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2395
by: Chris McKeever | last post by:
I am trying to modify the Mailman Python code to stop mapping MIME-types and use the extension of the attachment instead. I am pretty much clueless as to what I need to do here, but I think I have...
1
1652
by: Roman Yakovenko | last post by:
Hi. I need help( or solution :-) ). Problem: my project has 3 packages prj +------A +------B +------Utils The question is: what is the right way to use functionality from Utils in A and B...
9
1471
by: Tian | last post by:
I want to create a object directory called Context in my program, which is based on a dict to save and retrieve values/objects by string-type name. I have the definition like this: utils.py...
2
2247
by: BrianS | last post by:
Hi, I'm trying to learn Python and wanted to play with Tkinter. I couldn't get it to work so I figured it would help if I installed the newest verison of Python. I downloaded the source,...
1
3878
by: praba kar | last post by:
Dear All, In Php we can print RFC 2822 formatted date by date('r') with parameter r. Then it will print the below format date. "Thu, 7 Apr 2005 01:46:36 -0300". I want to print same RFC 2822...
5
2508
by: Ramon Diaz-Uriarte | last post by:
Dear All, Has anybody tried to use ID Utils (http://www.gnu.org/software/idutils/46) with Python? I've googled, searched the mailing list, and have found nothing. A silly, simple use of...
0
1609
by: Nico Grubert | last post by:
Hi there, I wrote a short python script that sends an email using python's email module and I am using Python 2.3.5. The problem is, that umlauts are not displayed properly in some email...
0
272
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 423 open ( +2) / 3539 closed ( +9) / 3962 total (+11) Bugs : 960 open ( -3) / 6446 closed (+20) / 7406 total (+17) RFE : 258 open...
0
1166
by: rkmr.em | last post by:
the memory usage of a python app keeps growing in a x86 64 linux continuously, whereas in 32 bit linux this is not the case. Python version in both 32 bit and 64 bit linux - 2.6.24.4-64.fc8 Python...
2
2626
by: Gabriel Rossetti | last post by:
Hello everyone, I'm trying to use python's freeze utility but I'm running into problems. I called it like this : python /usr/share/doc/python2.5/examples/Tools/freeze/freeze.py...
0
7119
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
6989
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
7157
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
1
6873
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
5453
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
1
4889
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
4579
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
0
3078
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
644
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.