473,699 Members | 2,568 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

SQLServer Table Partitioning

Hi!

I have a question:

I already have a DB that uses partitions to divide data in US
Counties, partitioned by state.

Can I use TWO levels of partitioning?

I mean... 3077 filegroups and 50 partition functions that address
them, but can I use another function to group the 50 states?

Thanks!

Piero

Jun 5 '07 #1
15 3679
Piero 'Giops' Giorgi (gi**********@g mail.com) writes:
I already have a DB that uses partitions to divide data in US
Counties, partitioned by state.

Can I use TWO levels of partitioning?

I mean... 3077 filegroups and 50 partition functions that address
them, but can I use another function to group the 50 states?
Do I understand it correctly that you already have 50 partitions, and
now you want even more? About what size do you expect per partition?

I'm not sure that partitioning by state is the best strategy. The partition
for Californina will be a lot bigger than the ones for Alaska and Rhode
Island.
--
Erland Sommarskog, SQL Server MVP, es****@sommarsk og.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx
Jun 5 '07 #2
Do I understand it correctly that you already have 50 partitions, and
now you want even more? About what size do you expect per partition?

I'm not sure that partitioning by state is the best strategy. The partition
for California will be a lot bigger than the ones for Alaska and Rhode
Island.
I know that, but partitioning by county makes the DB a lot easier to
maintain.
I have to work that way because I'm dealing with criminal records, and
they are separated by county with a ton of different files, so for
many of them I have to clear the table and reload the whole county
every time I get an update. Easier on partitions... :-)

Table size can be anywhere from 8000 to 3 million records, depending
on the county.

The best way to do that would be having a table partitioned over 3077
filegroups, so storing the data will go by COUNTY in this way:

CA_ALAMEDA
CA_ALPINE
CA_AMADOR
CA_BUTTE
CA_CALAVERAS
CA_COLUSA
CA_CONTRA_COSTA
CA_DEL_NORTE
CA_EL_DORADO
CA_FRESNO
CA_GLENN
CA_HUMBOLDT
CA_IMPERIAL
CA_INYO

With the COUNTY as the partition Parameter.

But, before trying, can I have 3077 files in ONE partition, and drop
all the states stuff?

Thanks!

Piero

Jun 5 '07 #3
Piero 'Giops' Giorgi (gi**********@g mail.com) writes:
I know that, but partitioning by county makes the DB a lot easier to
maintain.
I have to work that way because I'm dealing with criminal records, and
they are separated by county with a ton of different files, so for
many of them I have to clear the table and reload the whole county
every time I get an update. Easier on partitions... :-)

Table size can be anywhere from 8000 to 3 million records, depending
on the county.
Deleting 8000 rows is a breeze, but deleting 3 million rows takes
some resources, particularly if the rows are wide. But it still only
a matter of minutes.
But, before trying, can I have 3077 files in ONE partition, and drop
all the states stuff?
No, in the topic for CREATE PARTITION FUNCTION, I found that you
cannot have more than 999 boundary values.
--
Erland Sommarskog, SQL Server MVP, es****@sommarsk og.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx
Jun 5 '07 #4
On Jun 5, 3:24 pm, Erland Sommarskog <esq...@sommars kog.sewrote:
>But, before trying, can I have 3077 files in ONE partition, and drop
all the states stuff?
No, in the topic for CREATE PARTITION FUNCTION, I found that you
cannot have more than 999 boundary values.
Dang it... I kew there was a catch.

So, I'll be forced to have 3077 filegroups, grouped with 50
partitions.
Is there a way to have a partition function/scheme that sees other
schemes, instead of filegroups?

I mean Filegroups Counties (3077) - grouped by state (50) - all
together in ONE partitioned table.

Any Ideas?

Thank you!

Piero

Jun 6 '07 #5
Erland Sommarskog wrote:
Piero 'Giops' Giorgi (gi**********@g mail.com) writes:
>I know that, but partitioning by county makes the DB a lot easier to
maintain.
I have to work that way because I'm dealing with criminal records, and
they are separated by county with a ton of different files, so for
many of them I have to clear the table and reload the whole county
every time I get an update. Easier on partitions... :-)

Table size can be anywhere from 8000 to 3 million records, depending
on the county.

Deleting 8000 rows is a breeze, but deleting 3 million rows takes
some resources, particularly if the rows are wide. But it still only
a matter of minutes.
I do assume that (state, county) is an index. If not, then get
that fixed yesterday.
Jun 6 '07 #6
So, I'll be forced to have 3077 filegroups, grouped with 50
partitions.
Is there a way to have a partition function/scheme that sees other
schemes, instead of filegroups?
Why do you need separate filegoups? It seems to me that the main purpose of
partitioning here is for manageability and all those files/filegroups only
add to administration complexity and wasted space.

You might consider a hybrid solution with 50 individual state tables
included in a partitioned view, with each state table partitioned by county.
This approach would leverage partitioning to quickly reload individual
counties yet provide a seamless view of the entire country.
--
Hope this helps.

Dan Guzman
SQL Server MVP

"Piero 'Giops' Giorgi" <gi**********@g mail.comwrote in message
news:11******** **************@ j4g2000prf.goog legroups.com...
On Jun 5, 3:24 pm, Erland Sommarskog <esq...@sommars kog.sewrote:
>>But, before trying, can I have 3077 files in ONE partition, and drop
all the states stuff?
>No, in the topic for CREATE PARTITION FUNCTION, I found that you
cannot have more than 999 boundary values.

Dang it... I kew there was a catch.

So, I'll be forced to have 3077 filegroups, grouped with 50
partitions.
Is there a way to have a partition function/scheme that sees other
schemes, instead of filegroups?

I mean Filegroups Counties (3077) - grouped by state (50) - all
together in ONE partitioned table.

Any Ideas?

Thank you!

Piero
Jun 6 '07 #7
You might consider a hybrid solution with 50 individual state tables
included in a partitioned view, with each state table partitioned by county.
This approach would leverage partitioning to quickly reload individual
counties yet provide a seamless view of the entire country.
THANK YOU!
That is exactly what I want to do, but unfortunately I'm not (YET)
able to do it.

How can I have a partitioned view of partitioned tables?
I have the 50 state tables partitioned by county, but I can't get to
the next step.

Can someone post a small example of the thing?

Thanks

Piero

Jun 6 '07 #8
On Jun 5, 10:02 pm, Ed Murphy <emurph...@soca l.rr.comwrote:
I do assume that (state, county) is an index. If not, then get
that fixed yesterday.
Of course!
Actually it was fixes the day BEFORE yesterday... :-)

Piero

Jun 6 '07 #9
Piero 'Giops' Giorgi (gi**********@g mail.com) writes:
>You might consider a hybrid solution with 50 individual state tables
included in a partitioned view, with each state table partitioned by
county. This approach would leverage partitioning to quickly reload
individual counties yet provide a seamless view of the entire country.

That is exactly what I want to do, but unfortunately I'm not (YET)
able to do it.

How can I have a partitioned view of partitioned tables?
I have the 50 state tables partitioned by county, but I can't get to
the next step.

Can someone post a small example of the thing?
To me that sounds like a managability nightmare. While you can query
the beast in one query, when you need to flush the rows for Orange
County, you would have to explicitly to go to the CA table to
switch partitions, which would mean a lot of dynamic SQL.

I don't know if there is any catch with partition views over partitioned
tables (I really need to find some time to play with partitioned tables
to learn them!), but in a normal partitioned view you would have:

CREATE TABLE CA (state char(2) DEFAULT 'CA' CHECK (state = 'CA'),
-- other columns
PRIMARY KEY (state, county, whatever))

CREATE TABLE RI (state char(2) DEFAULT 'RI' CHECK (state = 'RI'),
...

CREATE VIEW thewholebunch AS
SELECT state, county, .....
FROM CA
UNION ALL
SELECT state, county, .....
FROM RI
....

But personally I would look into make the merging of new files more
effective than just dropping all existing rows.

--
Erland Sommarskog, SQL Server MVP, es****@sommarsk og.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx
Jun 6 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

18
6355
by: Jeff Boes | last post by:
I'm sure this is a concept that's been explored here. I have a table (fairly simple, just two columns, one of which is a 32-digit checksum) with several million rows (currently, about 7 million). About a million times a day we do select * from my_table where md5 = ? to verify presence or absence of the row, and base further processing on that information.
1
2050
by: Mats Kling | last post by:
Hi all, We are logging approx. 3 million records every day into a history table. Last week we ran into the 64 GB limit in UDB 8 so we recreated the table with 8 k pagesize to get some breathingroom before we hit the 128 GB limit. We are considering partitioning and I just wanted to check with you that our proposal is the best one:
2
8837
by: jorge | last post by:
I have just started working with DB2 Data Partition Feature.. Given a table name, how can I find out the partitioning key of that table? Is there a SYSIBM.* table that I can look at? Thanks, Jorge
10
2385
by: Sumanth | last post by:
Hi, I have a table that I would like to partition. It has a column c1 which has 100 distinct values. I was planning to partition the table on column c1 using a partioned index, and then apply data partitioned secondary indexes on the table. I then read about partioned table spaces, How do I get the same behaviour as above by creating a partioned table space?Do I create the partion table space, create the
2
2178
by: shsandeep | last post by:
Hi all, I have heard and read this many times: "Partitions should only be used for 'very large' tables". What actually determines whether a table is 'very large' or not? I have tables containing 0.5 million rows, 8 million rows, 14 & 29 million rows as well. How do I categorize them? Any comments will be helpful.
6
4116
by: shsandeep | last post by:
Can I partition an existing table? I have a table which has started growing at a very fast pace and would like to partition it. Thanks. Cheers, San.
10
3556
by: shsandeep | last post by:
DB2 V8.2 (not Viper yet and no range partitioning!!) I have created a table T1 (col1, col2) with col1 as the primary key. When I try to create a partitioning key on col2, it gives me error that it should have all primary keys included. So, I created table T1 again with col2 as the partitioning key. Now, I do not have col1 as the primary key. When I try to create col1 as the primary key, I get the following error: 1 The primary key, each...
9
17556
by: Veeru71 | last post by:
Can someone point me to good documentation on 'WITH clause" ? (I couldn't get much out of Queries section from SQL Reference manual). We are getting better performance when we explicity use global temp tables to store intermediate results than using "WITH cluase" in our queries. Where does DB2 store the intermediate results if the query uses "WITH clause" ? Thanks
2
2015
by: mandor | last post by:
Hello, I need some advise in table design, and more specifically about table partitioning. I read some papers and there was mentioned that if a table is expected to hold millions of rows, it's a good idea to partition it. Vertical partitioning, as I understood it, is separating data that differs in some way in a separate table, adding a key field as an identifier to what segment it belongs. The particular table holds signal measurements...
0
8623
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9197
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9054
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8941
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8897
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7785
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6549
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5881
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
2
2362
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.