473,657 Members | 2,461 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

help optimizing transpose query

58 New Member
Hello All!
Hoping some folks could help me optimize and or choose the best route to do this process.
First off, here is what I am trying to achieve. I have a (fairly large) table of ~34million rows that looks something like this.

ID,TIME,SPD
1,1,35
1,2,25
1,3,34
1,4,25
2,1,25
2,2,11
2,3,56
3,2,22
3,3,25
3,4,22

I am trying to transpose the table by ID into a new table that looks like this (although there will be 96 time fields)

ID,TIME1,TIME2, TIME3,TIME4
1,35,25,34,25
2,25,11,56,null
3,null,22,25,22

This was the original query I used (which worked fine when the table was 1.4 million rows, but now doesn't... it runs out of tempspace (even at 100mb tempspace size)

Expand|Select|Wrap|Line Numbers
  1. CREATE TABLE trans_data parallel 8 AS
  2. SELECT /*+ parallel (l,8) */
  3.   l.ID,
  4.   tbl_TIME1.TIME1,tbl_TIME2.TIME2,tbl_TIME3.TIME3,tbl_TIME4.TIME4
  5. FROM org_data l 
  6.   LEFT JOIN
  7.     (
  8.     SELECT
  9.       ID,
  10.       SPD AS TIME1
  11.     FROM
  12.       org_data l
  13.     WHERE
  14.       TIME=1
  15.     )
  16.      tbl_TIME1
  17.   ON
  18.     l.ID=tbl_TIME1.ID
  19.   LEFT JOIN
  20.     (
  21.     SELECT
  22.       ID,
  23.       SPD AS TIME2
  24.     FROM
  25.       org_data l
  26.     WHERE
  27.       TIME=2
  28.     )
  29.      tbl_TIME2
  30.   ON
  31.     l.ID=tbl_TIME2.ID
  32.   LEFT JOIN
  33.     (
  34.     SELECT
  35.       ID,
  36.       SPD AS TIME3
  37.     FROM
  38.       org_data l
  39.     WHERE
  40.       TIME=3
  41.     )
  42.      tbl_TIME3
  43.   ON
  44.     l.ID=tbl_TIME3.ID
  45.   LEFT JOIN
  46.     (
  47.     SELECT
  48.       ID,
  49.       SPD AS TIME4
  50.     FROM
  51.       org_data l
  52.     WHERE
  53.       TIME=4
  54.     )
  55.      tbl_TIME4
  56.   ON
  57.     l.ID=tbl_TIME4.ID;
this code worked about the same as far as speed little bit faster.. but again on the new larger table it runs out of tempspace.

Expand|Select|Wrap|Line Numbers
  1. CREATE TABLE CREATE TABLE trans_data parallel 8 AS
  2. SELECT /*+ parallel (l,8) */
  3. a.tmc,
  4. MAX(CASE WHEN a.TIME=1 THEN a.AVG END) AS TIME1,
  5. MAX(CASE WHEN a.TIME=2 THEN a.AVG END) AS TIME2,
  6. MAX(CASE WHEN a.TIME=3 THEN a.AVG END) AS TIME3,
  7. MAX(CASE WHEN a.TIME=4 THEN a.AVG END) AS TIME4
  8. FROM
  9. org_data a
  10. GROUP BY
  11. a.ID;
and here is my latest attempt
I create the table first, populate it with unique ID's then run this

Expand|Select|Wrap|Line Numbers
  1. UPDATE trans_data a SET TIME1 = (SELECT /*+ index (trans_data, ID_INDEX) */ AVG FROM orig_data b WHERE TIME=1 and a.ID=b.ID);
  2. COMMIT;
this works but takes about 8-10 minutes per column
which at 96 columns runs to over 8 hours, trying to shorten up the amount of time taken (if possible!)
Sorry for posting such a long post here, but if anyone can help, I will forever be indebted!
Cheers,
Eric
Mar 12 '08 #1
2 3920
Dave44
153 New Member
I like the second approach best personally, creating the pivot table seems is good. Is there no way to restrict on the rows? I mean why do you need to keep redoing the rows you did before? (there is no where clause in the query so your redoing the entire table!)

if you have built a new table from the old can you not add some kind of timestamp to it so you know what you have done already?
Mar 13 '08 #2
erbrose
58 New Member
Thanks for the reply Dave44
was a bit frustrated yesterday and not thinking as clearly as I should have been, your post gave me the much needed inspiration ie adding a where statement.
so just broke up the case statements into 4 temp tables by adding a where time between # and #
and then did a left join of the four temp tables at the end. It ran in less then 5 minutes...
Thanks
Eric
Mar 13 '08 #3

Sign in to post your reply or Sign up for a free account.

Similar topics

6
2000
by: Uros | last post by:
Hello! I have some trouble getting good results from my query. here is structure stat_views id | integer id_zone | integer created | timestamp
6
4342
by: ti33m | last post by:
Hi All, I'd like to include a datasheet on my user interface but since I'm starting to run tight on space, I'd like to have a vertically-oriented datasheet (column 1 has labels, column 2 has values), i.e. a transposed datasheet or datasheet in column format. A vertical datasheet will look cleaner, eliminate the need to scroll across (for miles) and lay out bit more efficiently since I have some long labels. It seems like Access only...
4
2854
by: Alan Lane | last post by:
Hello world: I'm including both code and examples of query output. I appologize if that makes this message longer than it should be. Anyway, I need to change the query below into a pivot table query. I'm having trouble doing it. Help! Here is my code so far: Sub OldRegionQuery()
1
6422
by: jenny.rhodes | last post by:
Hello, Please can anyone guide me on how to transpose an access table where I have many records per id eg UserID Question Answer 1 1 a 1 2 d 1 3 c 1 4 d
1
2328
by: Jenny | last post by:
Hello, Please can anyone guide me on how to transpose an access table where I have many records per id eg UserID Question Answer 1 1 a 1 2 d
3
3118
by: Gerard Brunick | last post by:
My way is ugly. These has to be a better way. Thanks, Gerard
1
1763
by: Nitin | last post by:
I have two tables in sql database as follows. I need to generate the output given at the end. ======= Table1 ======= PlanName PlanType PlanDescription Sequence
1
1359
by: eskelies | last post by:
Below you will find the code that I have written. I am asking it to search for an Excel file. Once it finds the correct dated Excel file it opens it. From there it is in search of information. I can't determine if the macro is working without fixing the following date - 1 issue. Any assistance would be greatly appreciated. I am using Excel 2003. Dim FileDate, Cusip, Here, Today, There FileDate = InputBox("Enter File Date in yyyymmdd...
12
19836
by: jenniferhelen | last post by:
I am working with a query that has 6 columns and 101 rows; I would like to transpose the rows and columns. I have tried using a crosstab query but Access limits the row "fields" to 3 and this was not producing the desired output. I saw a comment in a thread about using a pseudo row field as a work around but I do not understand how this works. Another option I tried is from the following link and attempting method 2: ...
0
8392
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8305
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8726
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8503
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8603
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
6163
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5632
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4301
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
1604
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.