Dynamic or not?

[rishiyoor]

I'm trying to write a program that will find the distance between two
groups of points in space, which have cartesian co-ordinates X,Y and
Z.

I need to find the distances between each point in one group and every
point in the other group. So if group 1 has 6 points and group 2 had 8
points, I will calculate 6 x 8 = 48 distances. But I do not know the
number of points the user will want to use. It is typically between 50
and 500 in both groups combined, but may be more. Since the memory
required for the distances will be much larger than the points
themselves, I am wondering if I will have to allocate memory
dynamically or if there are easier ways of solving such problems in
Python.

Suggestions for amateur programmer will be appreciated.

[Steven D'Aprano]

On Wed, 12 Dec 2007 19:18:20 -0800, rishiyoor wrote:

I'm trying to write a program that will find the distance between two
groups of points in space, which have cartesian co-ordinates X,Y and Z.

I need to find the distances between each point in one group and every
point in the other group. So if group 1 has 6 points and group 2 had 8
points, I will calculate 6 x 8 = 48 distances. But I do not know the
number of points the user will want to use. It is typically between 50
and 500 in both groups combined, but may be more. Since the memory
required for the distances will be much larger than the points
themselves, I am wondering if I will have to allocate memory dynamically
or if there are easier ways of solving such problems in Python.

Writing in Python, you never need to manually allocate memory. Sometimes,
for especially complicated data structures, you need to take care about
_de_allocating memory, but that's rare.

In this case, you may not have enough memory to calculate all the
combinations at once, so you should consider an iterator-based solution.
Read up on generators and iterators.

points1 = [ (1.2, 3.4), (5.7, 9.2) ]
points2 = [ (-6.3, 0.0), (14.1, -7.8), (2.6, 12.8) ]

import math
def distance(p1, p2):
return math.hypot(p1[0]-p2[0], p1[1]-p2[1])

def distances(list1, list2):
"""Yield the distances from every pair of points."""
for pt1 in list1:
for pt2 in list2:
yield distance(pt1, pt2)
Now, if you want the distances one at a time you do this:

>>D = distances(points1, points2)
for d in D:

.... print d
....
8.23468275042
17.0836178838
9.50368349641
15.1208465371
18.9620673978
4.75078940809
and if you want them all at once, you can do this:

>>list(distances(points1, points2))

[8.2346827504160718, 17.083617883809037, 9.5036834964133785,
15.120846537148639, 18.962067397834023, 4.7507894080878819]

--
Steven

[Tim Roberts]

ri*******@gmail.com wrote:

>
I'm trying to write a program that will find the distance between two
groups of points in space, which have cartesian co-ordinates X,Y and
Z.

I need to find the distances between each point in one group and every
point in the other group. So if group 1 has 6 points and group 2 had 8
points, I will calculate 6 x 8 = 48 distances. But I do not know the
number of points the user will want to use. It is typically between 50
and 500 in both groups combined, but may be more. Since the memory
required for the distances will be much larger than the points
themselves, I am wondering if I will have to allocate memory
dynamically or if there are easier ways of solving such problems in
Python.

Python already allocates memory dynamically. With 500 in each group,
you'll have 250,000 distances. If you use floating point, that's only 2
megabytes. Piece of cake. With 1,000 in each group, it's 8 megabytes.
Still no sweat.

What are you going to do with these distances? Why do you need them all in
memory?
--
Tim Roberts, ti**@probo.com
Providenza & Boekelheide, Inc.

[John Machin]

On Dec 13, 3:33 pm, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.auwrote:

On Wed, 12 Dec 2007 19:18:20 -0800, rishiyoor wrote:

I'm trying to write a program that will find the distance between two
groups of points in space, which have cartesian co-ordinates X,Y and Z.

points1 = [ (1.2, 3.4), (5.7, 9.2) ]
points2 = [ (-6.3, 0.0), (14.1, -7.8), (2.6, 12.8) ]

import math
def distance(p1, p2):
return math.hypot(p1[0]-p2[0], p1[1]-p2[1])

X, Y, .... umm, aren't we short a dimension?

[Steven D'Aprano]

On Thu, 13 Dec 2007 00:39:55 -0800, John Machin wrote:

X, Y, .... umm, aren't we short a dimension?

I'm not going to do *everything* for the OP. He can extend it to three
dimensions.
--
Steven

[rishiyoor]

On Dec 12, 11:33 pm, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.auwrote:

On Wed, 12 Dec 2007 19:18:20 -0800, rishiyoor wrote:

I'm trying to write a program that will find the distance between two
groups of points in space, which have cartesian co-ordinates X,Y and Z.

I need to find the distances between each point in one group and every
point in the other group. So if group 1 has 6 points and group 2 had 8
points, I will calculate 6 x 8 = 48 distances. But I do not know the
number of points the user will want to use. It is typically between 50
and 500 in both groups combined, but may be more. Since the memory
required for the distances will be much larger than the points
themselves, I am wondering if I will have to allocate memory dynamically
or if there are easier ways of solving such problems in Python.

Writing in Python, you never need to manually allocate memory. Sometimes,
for especially complicated data structures, you need to take care about
_de_allocating memory, but that's rare.

In this case, you may not have enough memory to calculate all the
combinations at once, so you should consider an iterator-based solution.
Read up on generators and iterators.

points1 = [ (1.2, 3.4), (5.7, 9.2) ]
points2 = [ (-6.3, 0.0), (14.1, -7.8), (2.6, 12.8) ]

import math
def distance(p1, p2):
return math.hypot(p1[0]-p2[0], p1[1]-p2[1])

def distances(list1, list2):
"""Yield the distances from every pair of points."""
for pt1 in list1:
for pt2 in list2:
yield distance(pt1, pt2)

Now, if you want the distances one at a time you do this:

>D = distances(points1, points2)
for d in D:

... print d
...
8.23468275042
17.0836178838
9.50368349641
15.1208465371
18.9620673978
4.75078940809

and if you want them all at once, you can do this:

>list(distances(points1, points2))

[8.2346827504160718, 17.083617883809037, 9.5036834964133785,
15.120846537148639, 18.962067397834023, 4.7507894080878819]

--
Steven

Thanks, that helps.

When you say python automatically allocates memory, what would you do
if you don't know the size of the list of, say for example, the
nearest pairs between the two groups. I would probably iterate over
all the pairs and create a new list. I did a similar operation in
another program, but I had to initialize the list (statically) with x
= [0]*50 before I could use it in the for loop.

[Marc 'BlackJack' Rintsch]

On Thu, 13 Dec 2007 06:51:23 -0800, rishiyoor wrote:

When you say python automatically allocates memory, what would you do
if you don't know the size of the list of, say for example, the
nearest pairs between the two groups. I would probably iterate over
all the pairs and create a new list. I did a similar operation in
another program, but I had to initialize the list (statically) with x
= [0]*50 before I could use it in the for loop.

Only if you used an index instead of starting with an empty list and
appending values to it. Or in simple cases a list comprehension might
replace a ``for`` loop.

"Bad" and "unpythonic" example:

new = [0] * len(old)
for i in xrange(len(old)):
new[i] = do_something(old[i])

Better written as:

new = list()
for item in old:
new.append(do_something(item))

Or as list comprehension:

new = [do_something(item) for item in old]

Or:

new = map(do_something, old)

Ciao,
Marc 'BlackJack' Rintsch

[Bruno Desthuilliers]

ri*******@gmail.com a écrit :

On Dec 12, 11:33 pm, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.auwrote:

(snip)

>
When you say python automatically allocates memory, what would you do
if you don't know the size of the list

thelist = []
thelist.append('Ever')
thelist.append('bothered')
thelist.append('reading')
thelist.append('the')
thelist.append('fine')
thelist.append('manual')
thelist.append('?')

print thelist

of, say for example, the
nearest pairs between the two groups. I would probably iterate over
all the pairs and create a new list. I did a similar operation in
another program, but I had to initialize the list (statically) with x
= [0]*50 before I could use it in the for loop.

You'd be better learning how to properly use lists and iterations in
Python... While you're at it, add list comprehensions,
iterators/generators and itertools to the list (pun intented).

HTH

[rishiyoor]

On Dec 13, 10:24 am, Marc 'BlackJack' Rintsch <[e-mail]:

On Thu, 13 Dec 2007 06:51:23 -0800, rishiyoor wrote:

When you say python automatically allocates memory, what would you do
if you don't know the size of the list of, say for example, the
nearest pairs between the two groups. I would probably iterate over
all the pairs and create a new list. I did a similar operation in
another program, but I had to initialize the list (statically) with x
= [0]*50 before I could use it in the for loop.

Only if you used an index instead of starting with an empty list and
appending values to it. Or in simple cases a list comprehension might
replace a ``for`` loop.

"Bad" and "unpythonic" example:

new = [0] * len(old)
for i in xrange(len(old)):
new[i] = do_something(old[i])

Better written as:

new = list()
for item in old:
new.append(do_something(item))

Or as list comprehension:

new = [do_something(item) for item in old]

Or:

new = map(do_something, old)

Ciao,
Marc 'BlackJack' Rintsch

Thanks Marc.

[John Machin]

On Dec 14, 2:29 am, Bruno Desthuilliers While you're at it, add list
comprehensions,

iterators/generators and itertools to the list (pun intented).

'Intented' is a novel word; is it the opposite of 'decamped'?