Hello,
I am trying to insert -9999 for any missing 10 minute data in a text file I am reading.
Data format sample is as follows in the input text file:
061201 1910 4.88 5.01
061201 1920 4.54 4.71
061201 1930 4.58 5.20
061201 2010 4.65 4.91
Required output:
061201 1910 4.88 5.01
061201 1920 4.54 4.71
061201 1930 4.58 5.20
061201 1940 -9999 -9999
061201 1950 -9999 -9999
061201 2000 -9999 -9999
061201 2010 4.65 4.91
I would appreciate any help. Thank you, RB
8 2033
Why aren't you including 1960-1990, and what happens if 1950 is the first record or the last record. Should the values always range from 1910-2010? - test_list = ["061201 1910 4.88 5.01",
-
"061201 1920 4.54 4.71",
-
"061201 1930 4.58 5.20",
-
"061201 2010 4.65 4.91" ]
-
-
test_dict = {}
-
first = ""
-
for rec in test_list:
-
substrs = rec.split()
-
test_dict[int(substrs[1])] = rec
-
first = substrs[0]
-
-
years_list = [1910, 1920, 1930, 1940, 1950, 2000, 2010]
-
for year in years_list:
-
if year in test_dict:
-
print test_dict[year]
-
else:
-
print "%s %d -9999 -9999" % (first, year)
Here's more detail: Col 0 is date (mmddyy), Col 1 is time (hhmm), cols 2,3 are wind speeds. Col 1 time and Col 0 date may start at any time whenever sensor starts measuring data. Once I fill in missing dates, times and values (-9999), then I'll go back and make hourly averages of the 10minute data - but I know how to do that.
061201 1910 4.88 5.01
061201 1920 4.54 4.71
061201 1930 4.58 5.20
061201 2010 4.65 4.91
Why would you want to use -9999 for missing values if you're going to calculate averages? It's going to throw your averages way off and will also give you negative averages.
While dwblas' code isn't exactly what you wanted, it contains the basic concept of what you're trying to accomplish. You should be able to use that as a starting point to move towards your end goal.
Thanks for the feedback - I'll deal with the -9999's in the average calculations loop.
bvdet 2,851
Expert Mod 2GB
Here's my suggested code: - data = '''061201 1910 4.88 5.01
-
061201 1920 4.54 4.71
-
061201 1930 4.58 5.20
-
061201 2010 4.65 4.91'''
-
-
# parse the data string to simulate reading in the file
-
dataList = [[s.strip() for s in item.split() if s.strip()] \
-
for item in data.split("\n")]
-
-
# create time list of 10 minute intervals based on first and last time
-
start = int(dataList[0][1])
-
stop = int(dataList[-1][1])
-
timeList = ["%s" % (start+i*10) for i in range((stop-start)/10+1)]
-
-
# create dictionary from dataList, then add missing data
-
dateStr = dataList[0][0]
-
dd = {}
-
for item in dataList:
-
dd[item[1]] = [item[i] for i in [0,2,3]]
-
for timeStr in timeList:
-
dd.setdefault(timeStr, [dateStr, "-9999", "-9999"])
-
-
# join data in dictionary for output
-
output = "\n".join([" ".join([dd[key][0],
-
key, dd[key][1],
-
dd[key][2]]) for key in timeList])
-
-
print output
Output: - >>> 061201 1910 4.88 5.01
-
061201 1920 4.54 4.71
-
061201 1930 4.58 5.20
-
061201 1940 -9999 -9999
-
061201 1950 -9999 -9999
-
061201 1960 -9999 -9999
-
061201 1970 -9999 -9999
-
061201 1980 -9999 -9999
-
061201 1990 -9999 -9999
-
061201 2000 -9999 -9999
-
061201 2010 4.65 4.91
-
>>>
Hello,
I'm trying to follow your (dwblas) code solution above but I need some comments along the way. Could you briefly comment each line please? Thanks, RB
bvdet,
I'm interested to know how you would change your solution above since column 2 is in the form of hhmm and so after 1950 the output needs to be 2000 and so on. And, after 2350, the date needs to change to 061202. How would you adjust for this? Thanks much! RB
bvdet 2,851
Expert Mod 2GB
In view of that requirement, the best option would be to go to the time module.
Convert the string "061201 1910" into a struct_time object, then convert to the number of seconds since the epoch. Add 10 minutes (600 seconds) and convert back to a string. - >>> import time
-
>>> stObj = time.strptime("061201 1910", "%y%m%d %H%M")
-
>>> stObj
-
(2006, 12, 1, 19, 10, 0, 4, 335, -1)
-
>>> seconds = time.mktime(stObj)
-
>>> seconds
-
1165021800.0
-
>>> newStr = time.strftime("%y%m%d %H%M", time.localtime(seconds+600))
-
>>> newStr
-
'061201 1920'
-
>>> stObj = time.strptime("061201 2350", "%y%m%d %H%M")
-
>>> time.strftime("%y%m%d %H%M", time.localtime(time.mktime(stObj)+600))
-
'061202 0000'
-
>>> stObj = time.strptime("061201 1950", "%y%m%d %H%M")
-
>>> time.strftime("%y%m%d %H%M", time.localtime(time.mktime(stObj)+600))
-
'061201 2000'
-
>>>
Sign in to post your reply or Sign up for a free account.
Similar topics
by: Justin Emlay |
last post by:
I'm hopping someone can help me out on a payroll project I need to
implement.
To start we are dealing with payroll periods. So we are dealing with an
exact 10 days (Monday - Friday, 2 weeks).
...
|
by: Wally |
last post by:
I have a record set (rs) that contains 25 barcodes values that I set to true
in a cookie. (see code section below)
If I read the cookie from within the same page that created it, I see all
25...
|
by: SQLJunkie |
last post by:
Here is an issue that has me stumped for the past few days. I have a
table called MerchTran. Among various columns, the relevant columns for
this issue are:
FileDate datetime
, SourceTable...
|
by: KC |
last post by:
Hello,
I am using Access 2002. WinXP, Template from MS called Orders Mgmt DB.
I have tweaked this DB to work for our small co.
It has worked pretty well up until I made the mistake of deleting...
|
by: Alexus12 |
last post by:
Crosstab is missing data from underlying table
while NO WHERE/HAVING clauses exist!
(Issue tested on Access97 SR1 and SR2):
this is WRONG results (jpeg shot):...
|
by: Chris |
last post by:
I finally got my report to be recongized, that login issue is terrible. Using
VS.NET 2003, created report inside of it. Now when I attempt to display the
report is blank (missing data, though I can...
|
by: danceli |
last post by:
After loading the BCP files that are created during the trigger/
reporting events I've noticed that the data in the table is missing
records. I've also noticed that the missing records (records in...
|
by: danceli |
last post by:
I have made trigger on table 'FER' that would be fired if data is
inserted, updated to the table. And also, I made batch file using bcp
to extract the newly updated / inserted records.
But I got...
|
by: anubis2k7 |
last post by:
Hi,
I am having a problem with dynamically sorting/grouping data in my report at runtime. My problem is that when the report is run using sorting/grouping I am missing data.
Specifically, my...
|
by: Mr.Tom.Willems |
last post by:
Hello people,
I am ussing an MS access database to enter and manage data from lab
tests.
until now i was the only one handeling the data so i had no need for a
controle on how missing data was...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
| |