Large Matrix into a hash of a hash

[pieceofcake]

Dear all,

I am new to Perl.

I have a large matrix. The easiest way to describe is it is like a calender where there are days along the top, numbers of weeks down the side, and the numbers of dates in the rest of the table.

Imagine this calendar to be populated with different data but with the same concept, but more like 10,000s across and 10,000s down. How would I parse this huge dataset into a hash. I know how to do the smaller example, as you can define the associative parts in the perl code. My file is too large for that. How do you define the top line and the side column into the hash as the important parts?

Kevin

[pieceofcake]

A bit more informations:-

I was reading the guidelines, and want to make it clear that I am not asking for code to be written for me. I am just confused as I am new and all the hash examples define the data within the perl code. I am just stating that my file is too large for that, so how do you define the keys and values, using the first row line and the first column. I am just confused by perl toy examples, as they do not show this step with reading a file in.

Your help will be very appreciated.

Kevin

[KevinADC]

use the open() function to open a file and start piping its contents into your perl script. http://perldoc.perl.org/perlopentut.html

[pieceofcake]

I have this which is for a 2 column file that has been fed into a hash:- as I was looking at examples of hashes.

#!/usr/bin/perl -w
 
use strict;
use warnings;
 
my $index1;
my $index2;
my %hashname;
my $hashname;
 
open (LIST1, "test.txt") || die "File not found\n";
     while (<LIST1>) {
          ($index1, $index2) = split(/\t/, $_);
          $hashname{$index1} = $index2;
 
          print $index1;
     }
close(LIST1);

But I want this type of script for a huge dataset with 10,000s of columns each way. So my index1 is my first column, and index2 is my row at the top, rather than the second column in this case.

Kevin

[KevinADC]

Show a sample of the data, with a few rows and columns and explain what you want to do. I don't understand what you said above: "So my index1 is my first column, and index2 is my row at the top, rather than the second column in this case".

[pieceofcake]

0 A B C D E F G --> thousands more
A 1 0 5 1 2 3 1
B 6 0 2 1 1 1 1
C 1 0 2 1 2 1 1
D 2 0 1 2 1 1 1
E 1 1 0 1 1 1 1
F 0 1 0 1 2 1 9
G 0 1 3 2 1 1 1

thousands more

I have to go through the hash of a hash, so the top line is one hash ref and the column down the side is another. I have to pick each number is over 2, and the letter ref of the row and column are stated rather than the number itself.

Example:
A D is 2 so more than 2 (A on top row, D in column)
C A is 5 so more than 2

need to print out A\tD
C\tA
and so on....

I am getting confused how to read it in properly. All the examples of HOH's are small toy examples (Barney, Fred etc....) that are loaded in the code rather than large datasets in a file.

Thanks,

Kevin

[KevinADC]

0 A B C D E F G --> thousands more
A 1 0 5 1 2 3 1
B 6 0 2 1 1 1 1
C 1 0 2 1 2 1 1
D 2 0 1 2 1 1 1
E 1 1 0 1 1 1 1
F 0 1 0 1 2 1 9
G 0 1 3 2 1 1 1

thousands more

I have to go through the hash of a hash, so the top line is one hash ref and the column down the side is another. I have to pick each number is over 2, and the letter ref of the row and column are stated rather than the number itself.

Example:
A D is 2 so more than 2 (A on top row, D in column)
C A is 5 so more than 2

need to print out A\tD
C\tA
and so on....

I am getting confused how to read it in properly. All the examples of HOH's are small toy examples (Barney, Fred etc....) that are loaded in the code rather than large datasets in a file.

Thanks,

Kevin

Unless there is some requirement that I am missing, I see no need for any complex data set to do this task. Process the input file line by line:

use strict;
use warnings;
open (LIST1, "test.txt") || die "File not found\n";
my $header = <LIST>; # read first line of file
chomp $header;
my @colms = split(/\s/,$header);
shift @colms; # remove the 0 in the first position so array starts with "A". 
# print "@colms\n"; 
while (<LIST>) {
   chomp;
   my @row = split(/\s/,$_);
   my $this_row = shift @row;
   for my $i (0..$#row) {
      next if $row[$i] < 2;
      print "$colms[$i]\t$this_row \n";
   }
}
close LIST;

Why do you want to use something like a hash of hashes?

[pieceofcake]

Hi Kevin,

Thank you for the code, it worked with my file with the correct output. Well done.

I just wanted to learn how to use a hash of a hash, I am learning perl and trying to practise different ways of solving a problem. To be honest with you, I have always used arrays in perl, and never a hash of hash before, so wanted to try it but for this example dataset I became confused.

I could understand the examples on the internet but they were always loading data within the script, I understand using easy toy examples, but it didn't help when it came to a huge file.

From looking at perl examples on the internet I could see how to split a row into, for example, 5 fields with this particular line:-

while( <FILE> ) {
( $firstname, $surname, $date, $age, $number ) = split( ':' );

I was confused how to do it for a row of 1000s, but I guess just loading it into an array was the best way.

Thank you for you help, it is very appreciated.

Kevin

[KevinADC]

OK, I see, you were more interested in the mechanics of building the data set read in from a file more than anything. If you look at thi spage:

http://perldoc.perl.org/perldsc.html

You will see headings that start with "Generation of....", they show how to build records from files. They don't show the open(FH, 'file.txt') part but just use while (<>) to show that some intput is coming in from a file. Of course they are rather simple examples but hopefully they help.