hi,
I have three arrays with variable values from a netCDF file. xy contains the vectors for the lon and lat coordinates. I am extracting the coordinates out of the two arrays (lon, lat) according to the vectors in the xy array. So far, I get the correct results, but the speed of the extraction process (for-loop) is extremely poor!!
Here's my code: -
from netCDF4 import Dataset
-
import scipy as scp
-
import numpy as np
-
-
# open netCDF file and create a dataset
-
rootfile = "myfile.nc"
-
rootgrp = Dataset(rootfile, "r")
-
-
# read variable values into arrays
-
xy = rootgrp.variables["zipxy"]
-
lon = rootgrp.variables["lon"]
-
lat = rootgrp.variables["lat"]
-
-
# create a new coordinates array
-
coordinates = np.array([[],[]])
-
-
# loop through the xy array and write the corresponding lon & lat coordinates
-
# into the coordinates array
-
# NOTE: the vectors in xy begin with [1,1] BUT the index of the values
-
# in lon & lat begins with [0,0] -> therefore: y-1, x-1
-
for x, y in xy:
-
coordinates = np.append(coordinates,[[lon[y-1,x-1]],[lat[y-1,x-1]]],1)
-
The dimensions are:
xaxis = 1320
yaxis = 1482
zip2 = 1080236
And the shape of the variables:
zipxy('zip2', 'two')
lon('yaxis', 'xaxis')
lat('yaxis', 'xaxis')
There must be a way to speed up the coordinate extraction (maybe I need another searching method through the arrays...). Until now, I couldn't find any solution.
Thanks for help!
3 2703 dwblas 626
Recognized Expert Contributor
A list/array is slow. Consider using a set or dictionary as they are hashed. It you just want all of the coordinates, a set made up of tuples=(x, y) or ([lat_x, lat_y], [lon_x, lon_y]) will work fine. Your time consumer is probably the lookups here so time the read/create arrays and the for loop separately so you know where the problem is. - # for x, y in xy:
-
# coordinates = np.append(coordinates,[[lon[y-1,x-1]],[lat[y-1,x-1]]],1)
Thanks a lot for the both answers!
I will try some possibilities and post the results
Sign in to post your reply or Sign up for a free account.
Similar topics |
by: Steven James Samuel Stapleton |
last post by:
Will calling ksort() on an array speed up it's access?
For example, I have the array $file_index, which is accessed by a key (the
entry id) and has two sub elements in a one dimensional array...
|
by: matthew.weiner |
last post by:
I have a bunch of SPs that all rely on a UDF that parses a comma
delimitted list of numbers into a table. Everything was working fine,
but now my application is growing and I'm starting to...
|
by: Hal Styli |
last post by:
Hello,
Can someone please help.
I need to use a large array and I'm not sure when one should
use (A) and when one should use (B) below:-
#define MAX 10000000
(A) int x;
|
by: Jason Huang |
last post by:
Hi,
To make it short, how do we do the data extraction to MSWord using ASP.Net
C#?
Any help will be appreciated.
Jason
|
by: Jason Huang |
last post by:
Hi,
Would someone show me how to do the data extraction to Excel in ASP.Net
using C# web form?
I am not familiar with VB, so I am asking someone to help me out!
Any help will be appreciated.
...
| |
by: Alastair Alexander |
last post by:
Hi ... I'm using pythoncom to create a python COM server application that
needs to be able to return large arrays to COM client apps. For example, I
need to be able to return an array to Excel that...
|
by: Peter Duniho |
last post by:
This is kind of a question about C# and kind of one about the framework.
Hopefully, there's an answer in there somewhere. :)
I'm curious about the status of 32-bit vs 64-bit in C# and the...
|
by: Stubert |
last post by:
I have a training module db that stores information about employees
and what training they have carried our or need to carry out. One
table in this database stores what training needs to be carried...
|
by: John |
last post by:
I have two large arrays. Is there a rapid method to comaprs the two
arrays and creat a third array of onlt thos items that the are in the
first array but not the second array. I do it manually...
|
by: hamishd |
last post by:
What is the best way to store large arrays of numbers (eg, 4-byte
integers)?
Say I want to store an array of 1billion 4-byte integers.. If my
computer has 4gB memory, then is this possible?
...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The...
| |
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |