473,651 Members | 2,716 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

.csv file - problem with data manipulation

4 New Member
Hello all;
I'm having trouble with my perl script and need some guidance. I am able to read data from a .csv file using the script and am placing it into an array @field. The data consists of four columns. I'm using the first column $field[0] and the fourth column $field[3], and placing that into a hash.

My problem is that I want to be able to use the data in $field[1], to reference the keys in the hash. $field[0] consists of number 1 - 4, and $field[1] consist of number 1-4 in random order.

I want my output to be the data in $field[3] corresponding to $field[0] (its row).

I've tried looping through the hash using :
while (($key,$value) = each(%hash), and have only been successful to output the data in $field[3] when $key is "= =" to $field[1].

Hope I explained the problem well enough, if any further explanation is necessary please let me know. I also have a portion of my code it necessary.

Thanks
Jul 17 '07 #1
6 1796
numberwhun
3,509 Recognized Expert Moderator Specialist
It would be a huge help if you could do two things:

1. Provide us with your script, as you have it so far. This will allow us to see all of your code, including modules that you may be using.
2. A sample of the file you are parsing with the script so we can see the data in question as well. Even if you don't post real data, it should be in the format the real data is.

Thanks!

Jeff
Jul 17 '07 #2
mojo
4 New Member
Sure no problem, just to warn you the code is a little messy because I've tried so many things, most of it is commented out just incase I need to reference back to it:

The data im using looks similar to this, but this is an example:

column 1 column 2 column 3
1 3 dog
2 2 cat
3 4 mouse
4 1 hat

I want the output to be outputed to a .csv file and look like this:

column 1 column 2 column 3 column 4
1 3 dog mouse
2 2 cat cat
3 4 mouse hat
4 1 hat dog

Thanks


Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl
  2. #use warnings;
  3.  
  4. open(INFILE,"perl_data.csv") || die ("Could not open file");
  5. open(OUT, ">test.csv") || die("Could not write file");
  6. open(IN, "test.csv") || die("Could not open file");
  7.  
  8.  
  9. my @field =(0);
  10. my $field;
  11.  
  12. my @created_array = (0 .. 4);
  13.  
  14. while (<INFILE>) {
  15.     s/\n//;
  16.     @field =split(",");
  17.  
  18.     my %hash = ($field[0],$field[3]);
  19.  
  20.  
  21. #    while ( my ($key, $value) = each(%hash) ) {
  22. #        $key = $field[1];
  23. #        if ($key != $field[0] || $key = $field[1]) {
  24. #            print "$key => $value\n";
  25. #        }
  26. #    }
  27.  
  28. #    foreach my $key (keys %hash) {
  29. #        print "$key = $hash{$key};
  30. #    }
  31.  
  32. #    my $feature;
  33. #    my @test_array = <IN>;
  34. #    print "$field[1]\n";
  35. #    foreach $value (@created_array) {
  36. #        my $value = $hash{@created_array};
  37. #        print $value;
  38. #    }
  39.  
  40. #    print "$field[0][0]\n";
  41. #    foreach my $key (keys %hash) {
  42. #        print "$key = $hash{$key}\n";
  43. #    }
  44.  
  45. #    foreach ($
  46. #        print "$#field";
  47. #        test_array = split(",");
  48.  
  49. #    foreach my $key (keys %hash) {
  50. #        $key = $field[1];
  51. #        print "$key = $hash{$key}\n";
  52. #    }
  53. #    foreach ($field[1]) {
  54. #        print OUT "$field[1]", ",";
  55. #    }
  56.  
  57. #    if ($field[1] > 0) {
  58. #        my $field1 = $compare{$field[1]};
  59. #        print "$field1\n";
  60. #    }
  61. #    print "$field[1]";
  62.  
  63. #    my @field1 = $compare{$field[1]};
  64. #    print "@field1";
  65. #    print "$field[1]";
  66. }
  67. close (INFILE);
  68.  
Thanks
Jul 17 '07 #3
mojo
4 New Member
Sorry about the mess heres a better idea of what im trying to do :

Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl
  2. #use warnings;
  3.  
  4. open(INFILE,"perl_data.csv") || die ("Could not open file");
  5. open(OUT, ">test.csv") || die("Could not write file");
  6. open(IN, "test.csv") || die("Could not open file");
  7.  
  8. my @field =(0);
  9. my $field;
  10. my @column1=(0);
  11. while (<INFILE>) {
  12.     s/\n//;
  13.  
  14.     @field =split(",");
  15.     my %hash = ($field[0],$field[3]);
  16.     while ((my $key, my $value) = each(%hash)) {
  17.         if($field[1] != $field[0]) {
  18.             $key = $field[1];
  19.             print "$field[0]","$field[1]","field[3]", "$hash{$key}\n"
  20.         } else {
  21.             print "$field[0]","$field[1]","field[3]", "field[3]\n"
  22.         }
  23.     }
  24.  
when I do this though only the data that match in column one and column two work..
Jul 17 '07 #4
KevinADC
4,059 Recognized Expert Specialist
You may need to follow this up with a tutorial in regards to references.

Expand|Select|Wrap|Line Numbers
  1. use warnings;
  2. use strict;
  3. #use Data::Dumper;
  4. my %hash;
  5. open(IN,"data.csv") || die ("Could not open file");
  6. while(<IN>){
  7.    chomp;
  8.    my @t = split(/,/);
  9.    $hash{$t[0]} = { parent => $t[2], child => $t[1] }; 
  10. }
  11. close(IN);
  12. #print Dumper \%hash;
  13. foreach my $key (sort {$a <=> $b} keys %hash) {
  14.    print join(',', $key,
  15.                    $hash{$key}{child},
  16.                    $hash{$key}{parent},
  17.                    $hash{$hash{$key}{child}}{parent} );
  18.    print "\n";
  19. }

I used "parent" and "child" really for my own benefit in figuring this out. No implication of a parent/child relationship is being implied, although there is similarity to a parent/child tree structure. Hopefully your real data is also applicable to this type of data structure.

The first three tutorials on this page discuss references:

http://perldoc.perl.org/index-tutorials.html
Jul 18 '07 #5
mojo
4 New Member
Thanks for the advice,I almost have the program working Now the problem is when I print the line I get the wrong output heres the code thus far:

Expand|Select|Wrap|Line Numbers
  1. open(INFILE,"perl_data.csv") || die ("Could not open file");
  2. open(OUT, ">test.csv") || die("Could not write file");
  3. open(IN, "test.csv") || die("Could not open file");
  4.  
  5. $counter =1;
  6. while(<INFILE>) {
  7.     chomp;
  8.     (@data1[$counter],@data2[$counter],@data3[$counter],@data4[$counter])= split(/,/);
  9.     $counter++;
  10. }
  11. close(INFILE);
  12.  
  13. for (my $i = 1; $i <= $#data1; $i++) {
  14.     $n = @data2[$i];
  15.     if(@data3[$i] >= .5) {
  16.         print @data4[$n],@data4[$i],"\n";
  17.         #print $data4[$i];#," ",@data4[$i]," ",@data1[$i],@data2[$n], "\n";
  18.     }
  19. }
  20.  
it wont let have @data4[$n] and @data[$i] on the same line???
I want it to print

boat cat ... but its not letting me print boat if i want cat also on the same line????
Jul 18 '07 #6
KevinADC
4,059 Recognized Expert Specialist
can't help without seeing the data you are using in your latest code.
Jul 19 '07 #7

Sign in to post your reply or Sign up for a free account.

Similar topics

10
9106
by: J. Campbell | last post by:
OK...I'm in the process of learning C++. In my old (non-portable) programming days, I made use of binary files a lot...not worrying about endian issues. I'm starting to understand why C++ makes it difficult to read/write an integer directly as a bit-stream to a file. However, I'm at a bit of a loss for how to do the following. So as not to obfuscate the issue, I won't show what I've been attempting ;-) What I want to do is the...
0
1424
by: Roland Hall | last post by:
I'm looking for information on working with large data files using FSO, XML. I have a program which creates a large CSV file, over 7mb. It's a rate table of freight shipping costs. There are certain fields I do not need, some are blank. A typical line would be: Raw data: " ", "30142", "GA", "01001"," ", "MA","
7
2045
by: Dennis C. Drumm | last post by:
I would like to be able to update an xml file located on my hosted server from my local computer. The server requires a user name and password to access the web site for writing or updating files. Could someone refer me to a doc that explains this or provide some sample code? Thanks,
4
1874
by: Sindarian | last post by:
This just seems like the most basic thing, but I can't find a simple description of this process anywhere and in here, everone is talking about going the other way :( I had .NET create an XML Schema file (xsd) for a table in my database. I then create a DataSet that contains 1 row of data from that same table. I want to use this DataSet along with the Schema file to create an XML file that ACTUALLY CONTAINS THE DATA. Simple huh: DataSet +...
2
3572
by: PFI | last post by:
Hi, I recently installed SQL Server 2005 Enterprise Edition (9.0.1399) and I have problems on queries concerning system file manipulation... For example, when I try to increase the Log File of MSDB database : ALTER DATABASE msdb MODIFY FILE (NAME = 'MSDBLog', SIZE = 50MB)
5
4571
by: kutty | last post by:
Hi All, I am loading data to a child table from a text file. the text files also contains data not referenced by parent key. while loading the data if one row fails to satisfies the constraint everything is getting rollback.. plz suggest me something.. which will help me to discard the unsatisfied rows and continue with the rest..
12
2884
by: SAL | last post by:
Hello, Is it possible to read a CSV from the Client, and bind my Datagrid to the data in the CSV file without uploading the file to the Server first? I have tried and in Debug mode on my workstation it works fine, but when I publish the page on our DEV server it doesn't fine the CSV file from the client. Has anyone done this before? If so, how do I do it? I'm new to ASP.net so
8
1929
by: rdemyan via AccessMonster.com | last post by:
Anyone have any ideas on how to determine when the back-end file (containing only tables) has been updated with new data. The date/time of the file won't work because it gets updated to the current date/time when the back-end file is compacted. I'm just looking for an easy way to determine when there has been a change to data in any table in the back-end file. Maybe something is updated in one of the system tables??
4
2550
by: for.fun | last post by:
Hi all, I have a really short time to perform a C++ development task and I need to : - find some string in a file - move some text data from one place of the file to another - replace some data in the file The problem is my file manipulations methods have to be robust and I
15
17372
by: pakerly | last post by:
How would i do this, convert a test file to excel? Lets say my text file has fields like this: NUMBER NAME ADDRESS PHONE 11002 Test1 TestAddress1 111-2222 lets say the number field is 10 characters long, there is a space and Name field is 15 characters long, there is a space, address can be 25 characters long, there is a space and phone is 10 characters long I want to...
0
8352
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8275
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
1
8465
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8579
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7297
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6158
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5612
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4144
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
1909
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.