473,394 Members | 1,755 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,394 software developers and data experts.

Save time processing netcdf

Hello,

I am working with a netcdf file and trying to remove a long string
from it e.g.
"KAST BLAH BLAH BLAH BLAH DATA BLAH DATA BLAH BLAH BLAH BLAH BLAH DATA
BLAH DATA BLAH"

Right now what I have works, but takes a long time to run... I think
this could be cut down to a quick 2 second run, but now takes around 5
minutes primarily because of the problem described below...

Why can't I use something like line 9 to get the entire string (which
returns [0], not
["KAST BLAH BLAH BLAH BLAH DATA BLAH DATA BLAH BLAH BLAH BLAH BLAH
DATA BLAH DATA BLAH"])? Rather I have to loop through it grabbing one
character at a time which drastically increases the processing time
and is very cumbersome.

Any one have any ideas or suggestions?
0 #OPEN NetCDF FILE TO READ
1 allDimNames=file.dimensions.keys()
2 max_string_length=file.dimensions['maxstringLen']
3 variableNames=file.variables.keys()
4 globalAttList=dir(file)
5 globalAttValue=getattr(file,'filePeriod')
6
7#GRABS DATA FROM STRING (2 DIMENSIONAL ARRAY) WITH THE CHARACTERS AS
INDIVIDUAL 8ELEMENTS
9 #YOU CAN'T JUST SPECIFY DATA VALUES DATA[0][0:max_string_length]
10 data=file.variables['rawstring'].getValue()
11 num_stations=data.shape
12 station_id=[str(mz)]
13
14 for m in station_id:
15 station_id=m
16 prec=[]
17 temps=[]
18
19 #Cycles through all of the stations in the file (LARGE ARRAY!!)
20 for k in range(num_stations[0]):
21 #Finds a certain station
22 if data[k][6]=='K' and data[k][7]==station_id[0:1] and data[k]
[8]==station_id[1:2] and 23data[k][9]==station_id[2:3] and data[k]
[15]=='5' or data[k][6]=='K' and data[k][7]==station_id[0:1] and
24data[k][8]==station_id[1:2] and data[k][9]==station_id[2:3] and
data[k][15]=='4':
25
26 #SEPARATES STRING CHARACTER BY CHARACTER ONLY WAY I'VE 27BEEN
ABLE TO FIGURE OUT HOW TO READ ENTIRE STRING INTO A STRING
28 for j in range(max_string_length):
29 prec.append(str(data[k][j]))
30 S= ''.join(prec)
31 # # # ##THEN RIP OFF THE WHITESPACE AT THE RIGHT OF THE
STRING
32 code=S.rstrip("\0")
33

Thanks for any of your help!

Mar 25 '07 #1
0 929

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: bandw | last post by:
I am having a problem using Numeric-24.0b2 in conjunction with the NetCDF module from ScientificPython (version 2.4.9). This problem does not surface using Numeric-23.8. The problem arises in...
6
by: Clay Beatty | last post by:
When you create database diagrams in Enterprise Manager, the details for constructing those diagrams is saved into the dtproperties table. This table includes an image field which contains most of...
1
by: elziko | last post by:
Does anyone have any experience in writing netCDF or HDF files using the ..NET framework (pref VB.NET) since all searches on the internet have turned up various C/C++ and Fortran libraries. TIA
9
by: skilpat | last post by:
I am going to be doing a lot of work with large data sets stored in various netCDF files, and after checking out the alternatives, I would really like to go with SciPy. The problem is that SciPy...
2
by: Luiz Vianna | last post by:
Hi folks, I got a problem that certainly someone had too. After a user request, I (my server) must process a lot of data that will expend some time. During this process I must inform the user...
4
by: xAvailx | last post by:
Hello: I didn't find any documentation that notes save point names are case sensitive, but I guess they are... Stored Proc to reproduce: /* START CODE SNIPPET */ If Exists (Select * From...
5
by: James Mitchelhill | last post by:
Sorry for the clunky subject line - I have a feeling that not knowing the proper terms for this is part of my problem. I'm trying to write a class that analyses some data. I only want it to do...
5
by: lars.uffmann | last post by:
Hi - since my Suse 10.1 didn't have netcdf support enabled by default, I downloaded the sources from http://www.unidata.ucar.edu/software/netcdf/ then did ../configure --prefix=/usr make make...
3
by: markon | last post by:
Hi, I would like to run few lines of code and limit the time. If the procedure would not be finished say in 1000ms I would like to stop processing it. Unfortunately I do not know where to...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.