Hi, I am writing a word stemming program, the program reads from a *.trec file to get query words in <title></title> tag, then stems the query words and output the result to another *.trec file. But perl builder promts "out of memory": -
use stem;
-
use WordNet::QueryData;
-
-
$wn = WordNet::QueryData->new;
-
$stemmer = stem->new($wn);
-
-
-
my $topic= "D:\\Program Files\\Perl Builder2\\MYPERL\\TrainingTopics.trec";
-
my $mytopic="D:\\Program Files\\Perl Builder2\\MYPERL\\myTopics.trec";
-
open FILE,"$topic" or die $!;
-
open MYTOPIC,">$mytopic" or die $!;
-
-
while(my $file=<FILE>)
-
{
-
if($file=~/<title>(.+?)<\/title>/)
-
{
-
print MYTOPIC &stemming($file);
-
-
}
-
else
-
{
-
print MYTOPIC $file;
-
}
-
}
-
-
-
-
sub stemming
-
{
-
my $in = shift;
-
my $out = '';
-
-
foreach my $token (split /[ ,.:;?!]/, $in)
-
{
-
-
my @stems = $stemmer->stemWord($token);
-
-
$out .= $stems[0] . ' ';
-
-
}
-
return $out;
-
}
-
close FILE;
-
close MYTOPIC;
-
Thanks a lot!
15 4456
Well, you aren't slurping the file into an array so the size of the file shouldn't make a difference so much.
It makes me wonder how much system memory you have and how many programs are running on your machine. If that it not it, then you will probably need to take this to the PerlBuilder forums as it would be outside of the realms of Perl coding.
Regards,
Jeff
**Note: are you trying to run this from Perl Builder? If so, why not run the script from the command line instead? I personally never trust a program to run my Perl script. I would rather the interpreter do it directly, without a middle man so to speak. As you can see, they always seem to find a way to cause issues.
Regards,
Jeff
Jeff.... Perl Builder is an IDE (its the one I use), it uses perl to run your perl programs same as any IDE.
That script looks like it would use very little memory so I don't know why its running out of memeory unless that module you are using has some known issues. WordNet::QueryData queries a database, that is possibly where the memory problem is happening. Not sure what to suggest though.
Jeff.... Perl Builder is an IDE (its the one I use), it uses perl to run your perl programs same as any IDE.
Oh, ok. Sorry, wasn't aware. I tend to hand code everything and run from the command line. If I do use an IDE, I use either Eclipse (with EPIC plugin for Perl) or Komodo. Both are awesome.
Well, now I know. :)
Oh, ok. Sorry, wasn't aware. I tend to hand code everything and run from the command line. If I do use an IDE, I use either Eclipse (with EPIC plugin for Perl) or Komodo. Both are awesome.
Well, now I know. :)
I'm with you, Numberwhun. Notepad++ to edit, cmd.exe to run.
Writing simple scripts in a text editor is fine, but using an IDE is essential to any complex code and even not so complex. You can run syntax checks, pipe data in/out of a script (command line or CGI), write and test code on the fly, use the perl debugger in the IDE, step over code, step into code, run timers , pause a script, see and jump to any subroutine in a single glance (may vary IDE to IDE) see all the variables used in a script (may vary IDE to IDE), and much more.
With the IDE you can open multiple scripts and select the one to run and check the output too, all on the fly if you wanted to.
Make your life easier and use an IDE.
In Perl Builder all the builtin perl functions display a basic syntax usage next to the function if you want it to. So you see something like:
split
(this displays in a little window next to the function)
split /PATTERN/,EXPR,LIMIT
split /PATTERN/,EXPR
split /PATTERN/
No need to look at the documentation if you foget the simple usage syntax. That right there is such a big time saver. And if you wanted to read the entire documentation you just click on the little window and a bigger window pops up with the full perl manpage for the function.
Wow, didn't mean to kick you up onto your soap box (j/k). I never said I didn't use an IDE. Komodo has always been my editor of choice but have been playing more with Epic (Eclipse) due to its interesting regex tester.
I agree with you that an IDE is a very useful part of Perl coding, but its not something you cannot live without. The software I support at work is all supported without an IDE, using GVim alone. Some of the files that we make additions to are several thousand lines or more in length.
Regards,
Jeff
i use Crimson Editor for my Perl Coding + other Languages. (FREE WARE)
i did use Perl Builder
Did I forget to mention I work for PerlBuilder? J/K..... I just had too much coffee this morning.
Did I forget to mention I work for PerlBuilder? J/K..... I just had too much coffee this morning.
*Dare I ask?* What IDE do you suggest? Is PerlBuilder free?
Jeff.... Perl Builder is an IDE (its the one I use), it uses perl to run your perl programs same as any IDE.
That script looks like it would use very little memory so I don't know why its running out of memeory unless that module you are using has some known issues. WordNet::QueryData queries a database, that is possibly where the memory problem is happening. Not sure what to suggest though.
It has no problem when I just convert one word to its stemming words, but when I try to stemming a number of words from a file, the "out of memory" occurs. I think its probably because the wordnet's database 2. I dont know much about how the database works, hope anyone can give me more suggestion.
PS: You guys are friendly and funny, thank you!
Since there is no WordNet 3.0 for windows, I download the WordNet 2.1 and change its name to WordNet 3.0... As I said, there was no problem to convert one word to its semming word.
I find a problem in my program. -
if($file=~/<title>(.+?)<\/title>/)
-
I thinks this sentence shoud pick the content between the tag <title></title>,
not containing the tag itself. But I test my program and I find the tag is contained in my first split word. For example, <title>perl bulider</title>, the first word is "<title>perl" not "perl"
The regexp is probably OK: -
my $file = "test <title>Perl Builder<\/title> test";
-
if ($file =~ /<title>(.+?)<\/title>/) {
-
print $1;
-
}
I think the problem is here: - foreach my $token (split /[ ,.:;?!]/, $in)
There is nothing in the above regexp (split) to split the string into tokens except a space. So it gets split into "<title>Perl" and "Builder<\/title>".
The regexp is probably OK: -
my $file = "test <title>Perl Builder<\/title> test";
-
if ($file =~ /<title>(.+?)<\/title>/) {
-
print $1;
-
}
I think the problem is here: - foreach my $token (split /[ ,.:;?!]/, $in)
There is nothing in the above regexp (split) to split the string into tokens except a space. So it gets split into "<title>Perl" and "Builder<\/title>".
I see your point, you are right! Thank you very much!!
I see your point, you are right! Thank you very much!!
You're welcome
Sign in to post your reply or Sign up for a free account.
Similar topics
by: henry |
last post by:
Hi,
Just wondering if there's a good GUI builder for Python. Something like
Visual Tcl, where you just drag and drop objects.
Not too good with creating user interfaces from code along.
...
|
by: John Smith |
last post by:
Hello,
I have a rather odd question. My company is an all java/oracle shop.
We do everything is Java... no matter what it is... parsing of text
files, messaging, gui you name it. My question...
|
by: Ziggi |
last post by:
Hi. I want to get a C++ IDE, but I dont know whether to go for Bill Gate's
solution or Borland's. Could any kind folks detail the relative strength
and weaknesses of both, and also tell me which...
|
by: Lotfi |
last post by:
Hi
I am trying to access MySQL DB with C++ Builder 6 pro,
I have Apache 2 under W2000 pro and MySQL 4
I found
http://crlab.com/mydac/
but it is not free :-(
but the trial version seems to...
|
by: Mihai Oltean |
last post by:
Hi
I wrote a C++Builder program that throws/catch many exceptions
(let's say 1.000.000/ hours). The problem is that I think that is a
bug in the Borland C++Builder exception handling mechanism....
|
by: P K |
last post by:
Hi,
I have around 30 fields on a screen each one of which I need to validate.
The idea was to have a set of messages to be displayed for all the posiible
errors instead of having to show a...
|
by: Kirt Loki Dankmyer |
last post by:
So, I download the latest "stable" tar for perl (5.8.7) and try to compile
it on the Solaris 8 (SPARC) box that I administrate. I try all sorts of
different switches, but I can't get it to compile....
|
by: Max58kl |
last post by:
Trying to access data and print it to the screen using Perl Builders I/O Window
--------------------------------------------------------------------------------
Hi
I am using a program called...
|
by: anklos |
last post by:
Hi, erveryone.
The default lib path on my perl builder is "c:/perl/lib". Recently, I install the WordNet-Similarity-2.05 and run the makefile in it, then run the following command:
make
make...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
| |