473,748 Members | 2,467 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Using hashes or arrays for file parsing

4 New Member
hi everyone,

I am kind of stuck and therefore would really appreciate some clues:

I actually have to run a script which has to compare two elements from two different files which are a blast file and a cdf file
I need also to keep the data structure
For this I chose the following strategy:

-dumping the files into two arrays
-doing a pattern matching between the two files.
-if it doesn't matches then remove the line.
-if the line has a different structure then keep the line

Here is the part of my script which take the most time
Expand|Select|Wrap|Line Numbers
  1.  
  2. foreach my $line(@CDF)
  3. {
  4.  
  5.     my $wanted;
  6.  
  7.         if ($line =~ /^.*?\t.*?\t.*?\t.*?\t.*?\t.*?\t.*?\t.*?\t.*?\t.*?\t.*?\t(.*?)\t/)
  8.         {
  9.             print "repeat again\n";
  10.             $wanted = ($1);
  11.             print $wanted."\n" ;
  12.             foreach my $lineB(@Blast)
  13.             {
  14.                 if ($lineB =~ /^($wanted)\s/)
  15.                 {
  16.                     print $wanted."\n";
  17.                     print OUTPUTFILEHANDLE "$line"; 
  18.                 }
  19.             } 
  20.         }
  21.  
  22.  
It takes hours to run it and obtain my output file.

Here are my questions:
Trying to only use subsets from the file instead of the complete 90Mb files
I have tried to use coordinate using array like this :

Expand|Select|Wrap|Line Numbers
  1.  
  2. my @array;
  3. print $array[0];
  4.  
  5.  
and then it ends up here printing the first line of the file...whereas I want 12th element of the line to do the comparison.

and also tried to understand hashes

So far I have read that it might be faster to use arrays than hashes therefore

Is there anyone who could give me some clue about how to define my file as a grid where I could use the coordinate x,y to get my subsets and then do my comparison?

I also though about using hashes to link key to values which would constitute the subsets I need but this way too I am stuck

I know that I could use the object oriented way but after having a look at it I think it is even more difficult so I would prefer to use one of the two previous methods

Any help is very welcome as I've been stuck for a while on this...
Jun 10 '08 #1
0 1325

Sign in to post your reply or Sign up for a free account.

Similar topics

35
3679
by: Troll | last post by:
Hi, I need to write a script which reads some data and reports the findings. Just to give you an idea the structure is similar to the following. Data input example: HEADING 1 ********** ColumnA ColumnB ColumnC ColumnD ColumnE
7
5657
by: Gary | last post by:
I haver a table of students - Say 100 students that I need to be able to update/delete and amend. I know I can do this one student at a time which is simple but lets say I want to see all the students on the screen at the same time, modify some, mark some for deletion and even have blank fields at the end to add a new record. In HTML which is generated I label each row and input field with a name/number combination i.e <input type=text...
10
14049
by: Randell D. | last post by:
Folks, I have a SELECT that returns with multiple records - It works when I have a LIMIT clause but when I try to include a GROUP BY clause, the select returns nothing (ie no records, no errors). The GROUP BY specifies a column that does exist in my db (and is also the only field that is select from the db). Where am I going wrong? Because my select uses hashes (which I provide with
9
1388
by: David Helgason | last post by:
I'm calling one stored procedure with a prepared statement on the server with 6 arrays of around 1200 elements each as parameters. The parameters are around 220K in total. This is taking a surprising amount of time. Thus I put a lot of logging into the application and in the stored procedure that's getting called. It seems that almost all of most of the time is spent before the stored procedure is even entered. What wakes my suspicion...
3
2387
by: RossettoeCioccolato | last post by:
Is there a brief tutorial somewhere on how to use the VC8 linker to generate a manifest for an isolated application with a dependency section for an arbitrary dll? There are some implementation details that I don't understand. Supposing I want to bind my application to a specific version of kernel32.dll using a hash, for example? What hash is used? Is it the hash generated by the CryptCATAdmin* functions or just a simple hash of the...
7
10287
by: christian.eickhoff | last post by:
Hi Everyone, I am currently implementing an XercesDOMParser to parse an XML file and to validate this file against its XSD Schema file which are both located on my local HD drive. For this purpose I set the corresponding XercesDOMParser feature as shown in the upcoming subsection of my code. As far as I understand, the parsing process should throw an DOMException in case the XML file doesn't match the Schema file (e.g. Element...
7
1372
by: RallyDSM | last post by:
Hello, I'm currently trying to read a .CSV file and get all the data into an array so I can work with it in the program. Here is what I currently have. Private Sub IntializeData() Dim AL As New ArrayList
3
1425
Chittaranjan
by: Chittaranjan | last post by:
Hi All, I am stuck with a problem in my coding that is the use of arrays and hashes in perl modules so any one can give me some link to good sites or examples to get deeper in array and hash in perl. Thanks, Chittaranjan
10
6556
by: aurekha | last post by:
Hi I have hashes with arrays to its keys like, %h1 = ('a'=>, 'b'=>, 'c'=> ); %h2 = ('a'=>, 'b'=>); then, how can i compare the 2 hashes(based on values. not keys) and get distinct values ?
0
8991
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8831
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
1
9326
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9249
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
6796
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4877
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3315
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2787
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2215
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.