file compare the hard way please help

[zmann1976]

Hi all this is my first post and I’m sorry I’m a noob.

I’ve been working on this for a couple of days and I cant seem to get it. I’m very sure that this is probably a very simple problem but it eludes me.

I need to do this in python on a Linux box. Here is the sequence of events.

Open a tcpdump file named something like “webdump.txt”, here is a sample of that file

11:30:07.830643 00:b0:64:19:86:f0 > 01:00:0c:cc:cc:cc snap ui/C len=35
0x0000: 0100 0ccc cccc 00b0 6419 86f0 0022 aaaa ........d...."..
0x0010: 0300 000c 2004 0100 0100 0500 0002 0005 ................
0x0020: 0400 0300 05a5 0004 000a 00b0 6419 86f0 ............d...
0x0030: 0001 42dc 861d 805c 0000 1400 ..B....\....
11:30:07.830722 00:b0:64:19:86:f0 > 01:00:0c:00:00:00 snap ui/C len=69
0x0000: 0100 0c00 0000 00b0 6419 86f0 0050 aaaa ........d....P..
0x0010: 0300 b064 0003 0084 0000 0100 0ccc cccc ...d............
0x0020: 00b0 6419 86f0 0032 aaaa 0300 000c 2004 ..d....2........
0x0030: 0100 0100 0500 0002 0005 0400 0300 05a5 ................
0x0040: 0004 000a 00b0 6419 86f0 0000 0000 0000 ......d.........
0x0050: 0000 0000 0000 0000 0000 30db e516 ..........0...

break the file up into each packet being in its own array, list, or container (for people who don’t work with tcpdump the packets start with the time stamp, and as you notice packets can be anywhere from a few lines to many lines) also removing the line end \n \t.
Next open another tcpdump file and find a match for each packet (that are now in arrays) in the first file, in the second file, if there is a match print match successful if there isn’t a match print the packet and match not found. I would have used filecmp but the information in file 1 and file 2 may be in a different order.
So far I have opened the file and put each packet into a stack. With

f = open('webdump.txt','rd')
for line in f.read().split('11:'):
stack = [line]
# stack.remove('\n')
print stack
f.close()
This doesn’t take the \n \t off (which I'm not sure is absolutely important as long as i can make a match)
and it also requires me to change the code every time i run it unless i only run my dumps in the 11 hour of the day. It also removes the 11: from the packet to look something like this:

['']
['30:07.830643 00:b0:64:19:86:f0 > 01:00:0c:cc:cc:cc snap ui/C len=35\n\t0x0000: 0100 0ccc cccc 00b0 6419 86f0 0022 aaaa ........d...."..\n\t0x0010: 0300 000c 2004 0100 0100 0500 0002 0005 ................\n\t0x0020: 0400 0300 05a5 0004 000a 00b0 6419 86f0 ............d...\n\t0x0030: 0001 42dc 861d 805c 0000 1400 ..B....\\....\n']
['30:07.830722 00:b0:64:19:86:f0 > 01:00:0c:00:00:00 snap ui/C len=69\n\t0x0000: 0100 0c00 0000 00b0 6419 86f0 0050 aaaa ........d....P..\n\t0x0010: 0300 b064 0003 0084 0000 0100 0ccc cccc ...d............\n\t0x0020: 00b0 6419 86f0 0032 aaaa 0300 000c 2004 ..d....2........\n\t0x0030: 0100 0100 0500 0002 0005 0400 0300 05a5 ................\n\t0x0040: 0004 000a 00b0 6419 86f0 0000 0000 0000 ......d.........\n\t0x0050: 0000 0000 0000 0000 0000 30db e516 ..........0...\n']

[bartonc]

Start by using

stack = line.split()

[zmann1976]

Figured it out with a bunch of help

import re
 
def getPackets(fileobj):
    packets = []
 
    for line in fileobj:
        #match the tcpdump timestamp
        pattern = '[0-9][0-9]:[0-9][0-9]:[0-9][0-9].[0-9][0-9][0-9][0-9][0-9][0-9] '
        # if line matches the pattern
        if re.match(pattern, line):
            # remove the timestamp
            line = re.sub(pattern, '', line)
            # add the line to the end of the list          
            packets.append(line)
        else: 
            # otherwise append to the end of the last
            # item in the list
            try: packets[len(packets)-1] += line
            # exception shouldn't occur, unless 
            # the input file is bad 
            except: pass
    return packets
 
list1 = getPackets(open('file1.txt'))
list2 = getPackets(open('file2.txt'))
 
# loop through list1, checking 
# if each of its elements is in
# list2
 
for packet in list1:
    if packet in list2:
        print 'Match Successful'
    else:
        print 'Match Unsuccessful'
        print packet