473,390 Members | 1,285 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,390 software developers and data experts.

How to link foreign keys & primary keys using python?

Hi all,
I hv started with python just recently... and have been assigned to
make an utility which would be used for data validations...
In short we take up various comma separated data files
for eg: area.txt, school.txt, students.txt.... and so on (ok?!?)
now,
1. area code used in the school.txt must be defined in the area.txt
(Primary key in area => area_code defined in area.txt
& Foreign key on school => area_code defined in school.txt)

i hv created primary key using the following piece of code:

# primary key for area.txt (index created on the column AREACODE as per
the
# schema defined earlier )
area_pk = PartitionedPK( name = 'Area PK ',
save_to =
'../Index/area_code.idx',
fields = ['A_AREACODE'] )
# col name in area schema

# foreign key is defined as follows...
school_fk = HashedFK( name = ' School code FK',
load_from = '../Index/area_code.idx',
fields = ['S_AREACODE'] )
# col name in school schema

Description for abv code:
1. An index {area_code.idx } is formed on the field AREACODE in the
area.txt
(i.e.,A_AREACODE)
2. The data values in the S_AREACODE field in the school.txt are
checked in the index {area_code.idx}
If the area code given in school.txt is not present in the area
code, then the record is not validated(as foreign key constraint
fails.)

Now if the Primary key is on mutiple columns...the foreign key, which
is also definedon the same no. of columns works..
for eg..

# primary key for school.txt (index created on the columns AREACODE &
SCHOOLCODE as per the
# schema defined earlier )
school_pk = PartitionedPK( name = 'School PK ',
save_to = '../Index/school.idx',
fields =
['S_AREACODE','S_SCHOOLCODE'] ) # col names in school schema

# foreign key for students is defined as follows...
student_fk = HashedFK( name = ' STUDENT code FK',
load_from = '../Index/student.idx',
fields =
['STUD_AREACODE','STUD_SCHOOLCODE'] ) # col name in
student schema

Now if I hv to define foreign key on student.txt but with only one
field, school code, so as to make sure that the school name given in
the student.txt exists in the school.txt whatever be the area code...
I am unable to do this...

Say, if the AREACODE field is not present in the student.txt file....
then???
I tried using the foll code

student_fk = HashedFK( name = ' Student FK',
load_from = '../Index/school.idx',
fields = ['STUD_SCHOOLCODE'] )

Its showing an AttributeError : 'list' object has no attribute
'has_key'

I also tried making another index with only SCHOOLCODE field (another
PK for school.txt) and the foreign key on student.txt to be loaded from
this index...
it still shows the same error
That's may be becoz... one SCHOOLCODE may repeat often, with a
different AREACODE... so may be we need to index on distinct
SCHOOLCODES in school.txt...
how do i do that??

I hope i am not confusing the genious' ... plz help me...
till get my hands on python... :)

Jun 9 '06 #1
9 3882
MTD
Your post is confusing. Here is my advice: investigate the use of
dictionaries. Dictionaries can allow you to define data in the form {
key:data }, e.g.

{ area_code : area_data }

{ (area_code,school_code) : school_data }

{ (school_code,student_code) : student_data }

Jun 9 '06 #2
"sonal" <so*******@gmail.com> writes:
Hi all,
I hv started with python just recently... and have been assigned to
make an utility which would be used for data validations... ,snip] plz help me... till get my hands on python... :)


1. This sure looks like a school assignment.

2. "till get my..." Do you actually not have python installed yet? It
is hopeless to tackle this problem if you haven't done a few simple
"hello, world" tasks.

3. Do you have a data model? In this case, you need to think
carefully about what a RDBMS, PK, and FK are. In otherwords, a
metamodel. Generally, once you understand the data structures you
should implement them pretty much verbatim in a "model" module.
Then do reader/writer modules so you can load that model from your
data sources and dump out to your data sinks.

4. Do you have a testsuite and test harness? Put together your test
harness, then develop for the simplest case, then add complexity.
E.g., no FK, FK with 1 attr, FK with multiple attrs, FKs with
shared attrs.
--
Harry George
PLM Engineering Architecture
Jun 9 '06 #3
Hi Mr. George,

Let me try it again...

I am not using any relational database to store the required tables
with primary keys & foreign keys....

When I say PRIMARY KEY =>
1. It means an index is created on the specified fields
(Out of various fields given in the comma separated txt file)
FileFormat: CODE, FIRST_NAME, last_name, area_of_expertise, country
Eg: A1,Harry,George, python, XYZCOUNTRY--------(1st record)

2. The index can be formed on a single field or on multiple fields
Eg: a. 'CODE' (single field ) {pk_code}
b. 'CODE' & 'NAME' (multiple fields ) {pk_code_fname}

Now when I say FOREIGN KEY =>
1. If the foreign Key is formed on the field 'CODE' in another text
file
Format: subsriber_code,CODE,first_name, no_of_posts,active(Y/N)
Eg: SUB_001, A1, Harry, 50, Y

This means the CODE (A1) given here is checked in the index formed
above
with primary key: pk_code...

2. If the foreign Key is formed on the fields 'CODE' & 'FIRST_NAME'
Format: subsriber_code,CODE,FIRST_NAME, no_of_posts,active(Y/N)
Eg: SUB_001, A1, Harry, 50, Y

This means the CODE (A1) & FIRST_NAME (Harry) given here
are checked in the index formed above with primary key:
pk_code_fname...

I am done till here.....

The problem starts if I have defined an index on multiple fields
(composite PK)
say: CODE & FIRST_NAME (pk_code_fname)
and if I need to define a FK on a single field out of these
say: CODE

I am unable to do that...
Mr. George, I thought i must explain the code i am dealin with,
for better understanding.., but i am sorry i confused you all the more
(incase, u want to view the code please refer to the code snippets in
my first query posted)

I hope you atleast get an idea of what i am hunting for.... :(

Jun 12 '06 #4

MTD wrote:
Your post is confusing. Here is my advice: investigate the use of
dictionaries. Dictionaries can allow you to define data in the form {
key:data }, e.g.

{ area_code : area_data }

{ (area_code,school_code) : school_data }

{ (school_code,student_code) : student_data }


Thanx Mr. Marc...
I am surely investigating the dictionaries... but the problem is that
I have to use the existing code...
and thas what is creating problems for me... :(

Jun 12 '06 #5
Hi Mr. George,
Sorry for confusing u so much...
Let me try it again...

I am not using any relational database to store the required tables
with primary keys & foreign keys....

When I say PRIMARY KEY =>
1. It means an index is created on the specified fields
(Out of various fields given in the comma separated txt file)
FileFormat: CODE, FIRST_NAME, last_name, area_of_expertise, country
Eg: A1,Harry,George, python, XYZCOUNTRY--------(1st record)

2. The index can be formed on a single field or on multiple fields
Eg: a. 'CODE' (single field ) {pk_code}
b. 'CODE' & 'NAME' (multiple fields ) {pk_code_fname}

Now when I say FOREIGN KEY =>
1. If the foreign Key is formed on the field 'CODE' in another text
file
Format: subsriber_code,CODE,first_name, no_of_posts,active(Y/N)
Eg: SUB_001, A1, Harry, 50, Y

This means the CODE (A1) given here is checked in the index formed
above
with primary key: pk_code...

2. If the foreign Key is formed on the fields 'CODE' & 'FIRST_NAME'
Format: subsriber_code,CODE,FIRST_NAME, no_of_posts,active(Y/N)
Eg: SUB_001, A1, Harry, 50, Y

This means the CODE (A1) & FIRST_NAME (Harry) given here
are checked in the index formed above with primary key:
pk_code_fname...

I am done till here.....

The problem starts if I have defined an index on multiple fields
(composite PK)
say: CODE & FIRST_NAME (pk_code_fname)
and if I need to define a FK on a single field out of these
say: CODE

I am unable to do that...
Mr. George, I thought i must explain the code i am dealin with,
for better understanding.., but i am sorry i confused you all the more
(incase, u want to view the code please refer to the code snippets in
my first query posted)

Thanks & regards,
sonal

Jun 12 '06 #6

MTD wrote:
Your post is confusing. Here is my advice: investigate the use of
dictionaries. Dictionaries can allow you to define data in the form {
key:data }, e.g.

{ area_code : area_data }

{ (area_code,school_code) : school_data }

{ (school_code,student_code) : student_data }


Thanx Mr. Marc...
I am surely investigating the dictionaries... but the problem is that
I have to use the existing code...
and thats what is creating problems for me... :(

Jun 12 '06 #7
so************@3i-infotech.com wrote:
Hi Mr. George,

Let me try it again...

I am not using any relational database to store the required tables
with primary keys & foreign keys....
None the less, you are using relational database terminology. If you
want people to understand you then you should use it correctly or
explain yourself in other ways ...
When I say PRIMARY KEY =>
1. It means an index is created on the specified fields
(Out of various fields given in the comma separated txt file)
FileFormat: CODE, FIRST_NAME, last_name, area_of_expertise, country
Eg: A1,Harry,George, python, XYZCOUNTRY--------(1st record)

2. The index can be formed on a single field or on multiple fields
Eg: a. 'CODE' (single field ) {pk_code}
b. 'CODE' & 'NAME' (multiple fields ) {pk_code_fname}

Now when I say FOREIGN KEY =>
1. If the foreign Key is formed on the field 'CODE' in another text
file
Format: subsriber_code,CODE,first_name, no_of_posts,active(Y/N)
Eg: SUB_001, A1, Harry, 50, Y

This means the CODE (A1) given here is checked in the index formed
above
with primary key: pk_code...

2. If the foreign Key is formed on the fields 'CODE' & 'FIRST_NAME'
Format: subsriber_code,CODE,FIRST_NAME, no_of_posts,active(Y/N)
Eg: SUB_001, A1, Harry, 50, Y

This means the CODE (A1) & FIRST_NAME (Harry) given here
are checked in the index formed above with primary key:
pk_code_fname...

I am done till here.....

The problem starts if I have defined an index on multiple fields
(composite PK)
say: CODE & FIRST_NAME (pk_code_fname)
and if I need to define a FK on a single field out of these
say: CODE

I am unable to do that...
Mr. George, I thought i must explain the code i am dealin with,
for better understanding.., but i am sorry i confused you all the more
(incase, u want to view the code please refer to the code snippets in
my first query posted)

I hope you atleast get an idea of what i am hunting for.... :(

You say you have to use existing code, which is a pity because it seems
that code is organised in such a way as to make your problem difficult.

The only ways to locate information by a *portion* of the primary key,
given that you appear to be using tuples as primary key values, is
either to maintain a separate index on the partial key (in other words
have a lookup table that holds the partial key values).

Note, also, that you run a risk here: there is no guarantee that only
one record will have any given partial key value.

The whole point of foreign keys in databases is that they are *complete*
primary key values, and can therefore refer at at most one occurrence of
the referenced entity. It is not correct to refer to a reference to a
partial primary key as a "foreign key", precisely because all guarantees
of uniqueness are lost.

It seems that you are not a native English speaker, so I am sorry if
this is leading to further problems.

The more simply you can explain yourself the easier it will be to help!

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Love me, love my blog http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

Jun 12 '06 #8
sonal wrote:
Hi Mr. George,
Sorry for confusing u so much...
Let me try it again...

I am not using any relational database to store the required tables
with primary keys & foreign keys....

When I say PRIMARY KEY =>
1. It means an index is created on the specified fields
(Out of various fields given in the comma separated txt file)
FileFormat: CODE, FIRST_NAME, last_name, area_of_expertise, country
Eg: A1,Harry,George, python, XYZCOUNTRY--------(1st record)

2. The index can be formed on a single field or on multiple fields
Eg: a. 'CODE' (single field ) {pk_code}
b. 'CODE' & 'NAME' (multiple fields ) {pk_code_fname}
What, in Python, *are* these indexes - lists, dictionaries, tuples or
something else?
Now when I say FOREIGN KEY =>
1. If the foreign Key is formed on the field 'CODE' in another text
file
Format: subsriber_code,CODE,first_name, no_of_posts,active(Y/N)
Eg: SUB_001, A1, Harry, 50, Y

This means the CODE (A1) given here is checked in the index formed
above
with primary key: pk_code...

2. If the foreign Key is formed on the fields 'CODE' & 'FIRST_NAME'
Format: subsriber_code,CODE,FIRST_NAME, no_of_posts,active(Y/N)
Eg: SUB_001, A1, Harry, 50, Y

This means the CODE (A1) & FIRST_NAME (Harry) given here
are checked in the index formed above with primary key:
pk_code_fname...

I am done till here.....

The problem starts if I have defined an index on multiple fields
(composite PK)
say: CODE & FIRST_NAME (pk_code_fname)
and if I need to define a FK on a single field out of these
say: CODE

I am unable to do that...
Mr. George, I thought i must explain the code i am dealin with,
for better understanding.., but i am sorry i confused you all the more
(incase, u want to view the code please refer to the code snippets in
my first query posted)

Thanks & regards,
sonal

A lot depends on the data structure that the code uses to represent the
"indexes". Perhaps you could explain how things are looked up in them?

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Love me, love my blog http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

Jun 12 '06 #9
Hi Mr. Steve,
The *indexes* i am using are lists...
The code for creation of the PartionedPK is given below...
************************************************** *****************************
class PartitionedPK(object):
def __init__(self, name, save_to, fields):
self.name = name
self.idx_name = save_to
self.part_name = save_to + PARTITION_SUFFIX
self.fields = fields
self.digester = sha.new

def setup(self, schema):
self.partitions = [[] for i in range(256)]
self.flush_pos = [[] for i in range(256)]
self.flush_count = 0
self.index = {}
self.offsets = field_offsets(self.fields, schema)
if not self.offsets:
raise ValueError('One or more index field names are
invalid')

self.idx_file = open(self.idx_name, 'wb+')
self.part_file = open(self.part_name, 'wb+')

def save(self):
pickle.dump(self.flush_count, self.part_file, -1)
pickle.dump(self.flush_pos, self.part_file, -1)
self.idx_file.close()
self.part_file.close()

def flush(self):
self.flush_count += 1
for i in range(256):
self.flush_pos[i].append(self.idx_file.tell())
pickle.dump(self.partitions[i], self.idx_file, -1)
self.partitions[i] = []

def valid(self, record, data):
key = self.digester(''.join( [data[i] for i in self.offsets]
)).digest()
self.partitions[ ord(key[0]) ].append( (key, record) )
# defer checking till later
return True

def finalize(self):
self.flush()
errors = []
for bin in range(256):
#show('Checking %s, bin %d/256 ... ' % (self.name, bin))
seen = {}
has = seen.has_key
for flush in range(self.flush_count):
self.idx_file.seek( self.flush_pos[bin][flush] )
records = pickle.load(self.idx_file)
for key, value in records:
if has(key):
errors.append(value)
else:
seen[key] = value
return errors
************************************************** *****************************
the PK definition is as follows:
vol_pk = PartitionedPK( name = 'VOL_PK',
save_to = '../Index/vol.idx',
fields = ['ID','Type','Curr_Code','Tenor'])

The code for the Foreign Key declaration (referencing to the indexes)
is as given below...
************************************************** *****************************
class HashedFK(object):
def __init__(self, name, load_from, fields):
self.name = name
self.filename = load_from
self.fields = fields
self.digester = sha.new

def setup(self, schema):
self.index = {}
self.offsets = field_offsets(self.fields, schema)

if not self.offsets:
raise ValueError('One or more index field names are
invalid')

file = open(self.filename, 'rb+')
self.index = pickle.load(file)
file.close()

def valid(self, record, fields):
key = self.digester(''.join( [fields[i] for i in self.offsets]
)).digest()

return self.index.has_key(key)

def flush(self):
pass

def finalize(self):
return None
************************************************** *****************************
the FK definition is as follows:
vol_fk = HashedFK( name = ' VOL_FK,
load_from = '../Index/vol.idx',
fields= ['ID','Type','Curr_Code','Tenor'])

The code is working fine when the foreign key is referenced to the
complete primary key

But if the FK were to be on only 'ID'
the FK defn would have been like =>

vol_fk = HashedFK( name = ' VOL_FK,
load_from = '../Index/vol.idx',
fields = ['ID'] )
This is were the problem lies...
it shows AttributeError: 'list' object has no attribute 'has_key'

I have also tried defining another PK with a single field as follows =>
Tvol_pk = PartitionedPK( name = 'TVOL_PK',
save_to = '../Index/tvol.idx',
fields = ['ID'] )
The index ''tvol.idx'' is being created at the given location(path
specified)
but referencing to this index(i.e., tvol.idx) with the vol_fk given
above also gives
the same error.

Jun 13 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
by: Bodza Bodza | last post by:
I'm having an argument with an incumbent self-taught programmer that it is OK to use null foreign keys in database design. My take is the whole point of a foreign key is that it's not supposed...
31
by: Robert Brown | last post by:
Let's say I have a type hierarchy: (just an example) the general entity customer: CREATE TABLE customer(customer_id int, customer_name varchar(250), customer_type int) three specific...
0
by: Scott Ribe | last post by:
I've got a problem which I think may be a bug in Postgres, but I wonder if I'm missing something. Two tables, A & B have foreign key relations to each other. A 3rd table C, inherits from A. A...
2
by: Ben | last post by:
Right now I have 1 table. The first part is the first and last name along with address etc. There is about 10-15 fields here. The second part consists of times, penalties and if they enter this...
1
by: Thomas T. Thai | last post by:
I'm looking for a better way to make use of foreign keys. Here is a sample setup: -- TESTING Foreign Keys create table mod ( mod_id int not null primary key, name varchar(32) not null...
2
by: kal stevens | last post by:
I have been trying to write a database schema in mysql, and I cant figure this out. Here is a database schema DROP DATABASE IF EXISTS d; CREATE DATABASE d;
1
by: rbarber | last post by:
I have to synchronize 2 databases hourly but am having difficulty maintaining foreign key relations. These tables use auto-increment columns as primary keys, with child records in other tables...
6
by: ravichoudhari | last post by:
i came accross requirement of multiple foreign keys in a table referencing the same primary key another table. i created the table relations using the relations editor in access. even though i...
4
by: Wolfgang Keller | last post by:
Hello, so far it seems to me as if the only ORM module for Python which supports composite primary/foreign keys was SQLAlchemy. Which looks a little bit "overbloated" for my needs: I "just" need...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.