Hi all,
I'd like to have you opinion about the following performance/design
issue:
a logging system is generating a lot of records every our,(about
500k/day)
I need to store them in a huge Oracle RAC system
a Java web application connects to the data and executes queries on
them, mainly on last entered set of records
queries on informations stored more then 3 months ago are quite rare
as first guess I managed to store them in a table indexed by a
primary key, let's say in the table 'LOGS'
the problem is that table is going to grows very big,
have you got 'best practice' ideas to handle the situation in a better
way?
I thought about moving data priodically in a 'LOGS_ARCHIVE' table ,
for example using a Oracle job
is it necessary?
PS I can't use Oracle standard file archiving support, I need data to
stay on tables
thanks in advance
jc
13  2259 
"curwen" <jo**********@d espammed.com> wrote in message
news:cc******** *************** ***@posting.goo gle.com... Hi all,
I'd like to have you opinion about the following performance/design
issue:
a logging system is generating a lot of records every our,(about
500k/day)
I need to store them in a huge Oracle RAC system
a Java web application connects to the data and executes queries on
them, mainly on last entered set of records
queries on informations stored more then 3 months ago are quite rare
as first guess I managed to store them in a table indexed by a
primary key, let's say in the table 'LOGS'
the problem is that table is going to grows very big,
have you got 'best practice' ideas to handle the situation in a better
way?
I thought about moving data priodically in a 'LOGS_ARCHIVE' table ,
for example using a Oracle job
is it necessary?
PS I can't use Oracle standard file archiving support, I need data to
stay on tables
thanks in advance
jc
I always find it strange that we see requests like this. Sort of along the
lines of "I am building this huge Oracle database and I don't know much
about Oracle but..." Wouldn't one get people on the project who have some
knowledge and expertise on Oracle? It just strikes me as very odd.
Jim
"Jim Kennedy" <ke************ *************** *@attbi.net> wrote in message news:<%Y3%b.560 84$Xp.266401@at tbi_s54>...
[] I always find it strange that we see requests like this. Sort of along the
lines of "I am building this huge Oracle database and I don't know much
about Oracle but..." Wouldn't one get people on the project who have some
knowledge and expertise on Oracle? It just strikes me as very odd.
Jim
My guess is the manager gets the cheapest programmer available and
keeps asking him to take over responsibilitie s of the more
knowledgable folks as they are laid off. Of course then the manager
hears about outsourcing to India at even lower rates and the pattern
begins again. 8^)
But seriously, I'm as puzzled as you are.
Ed
> > jc I always find it strange that we see requests like this. Sort of along the
lines of "I am building this huge Oracle database and I don't know much
about Oracle but..." Wouldn't one get people on the project who have some
knowledge and expertise on Oracle? It just strikes me as very odd.
Jim
My experience in computer science is limited, I only have a First
Class Masters degree in Electronics Engineering, but I'm confident
with hard working and studying I can become 'good' as you
plus, I'll be much more professional answering in Usenet
thanks anyway
jc ed********@magi cinterface.com (Ed prochak) wrote in message news:<4b******* *************** ****@posting.go ogle.com>... "Jim Kennedy" <ke************ *************** *@attbi.net> wrote in message
My guess is the manager gets the cheapest programmer available and
keeps asking him to take over responsibilitie s of the more
knowledgable folks as they are laid off. Of course then the manager
hears about outsourcing to India at even lower rates and the pattern
begins again. 8^)
But seriously, I'm as puzzled as you are.
Ed
My experience in computer science is limited, I only have a First
Class Masters degree in Electronics Engineering, but I'm confident
with hard working and studying I can become 'good' as you
plus, I'll be much more professional answering in Usenet
thanks anyway
jc
"curwen" <jo**********@d espammed.com> wrote in message
news:cc******** *************** ***@posting.goo gle.com... jc
I always find it strange that we see requests like this. Sort of along
the lines of "I am building this huge Oracle database and I don't know much
about Oracle but..." Wouldn't one get people on the project who have
some knowledge and expertise on Oracle? It just strikes me as very odd.
Jim
My experience in computer science is limited, I only have a First
Class Masters degree in Electronics Engineering, but I'm confident
with hard working and studying I can become 'good' as you
plus, I'll be much more professional answering in Usenet
thanks anyway
jc
I am sure you are a hard working good person. My comment has nothing to do
with your personality, but rather why people do this. (more the fault of
the manager from not getting experienced people for a task that needs an
experienced person) It is sort of like I've been hired to design a Boeing
747 and I have experience building paper airplanes. There is quite a gulf
there in needed experience.
Jim
Jim,
I do have some experience, maybe this is the most complex project I
have dealt with, but we have enough time and resources to improve the
skills if required..
anyway the project is not so mission critical, even if a lot of data
are expected, and from a general point of view the factors are quite
calibrated
anyway we used range partitioning to handle the issue
best regards
jc
"Jim Kennedy" <ke************ *************** *@attbi.net> wrote in message news:<HgI%b.298 33$AL.531756@at tbi_s03>... "curwen" <jo**********@d espammed.com> wrote in message
news:cc******** *************** ***@posting.goo gle.com... > jc
I always find it strange that we see requests like this. Sort of along the lines of "I am building this huge Oracle database and I don't know much
about Oracle but..." Wouldn't one get people on the project who have some knowledge and expertise on Oracle? It just strikes me as very odd.
Jim
My experience in computer science is limited, I only have a First
Class Masters degree in Electronics Engineering, but I'm confident
with hard working and studying I can become 'good' as you
plus, I'll be much more professional answering in Usenet
thanks anyway
jc
I am sure you are a hard working good person. My comment has nothing to do
with your personality, but rather why people do this. (more the fault of
the manager from not getting experienced people for a task that needs an
experienced person) It is sort of like I've been hired to design a Boeing
747 and I have experience building paper airplanes. There is quite a gulf
there in needed experience.
Jim
Hi
Engineer from india here :-)..Well there are many different ways to
ask a question and depending on the way its asked you would get many
different responses :-).
Your problem can it be understood this way you want to better manage
archival data that keeps on growing but is huge and you need it
online.
Well one possible solution would be to dig into the oracle Docs and
look into partitioning.Wh ich is divide and conquer approcah for
managing data .Specifically look at range partitioning.
well if you want to be sure that usenet advice really works for you
you should take a hint and start doing test for yourself.You are smart
hmm and you will take every advice offered here and test it out.
And if you are too lazy well indian engineer here would welcome you
with open arms ..remember outsourcing is just a arms length away. :-))
regards
Hrishy
P.S:Please keep outsourcing and other bull shit away from this news
group folks.
I hate it this forum is meant for oracle and technical discussions. jo**********@de spammed.com (curwen) wrote in message news:<cc******* *************** ****@posting.go ogle.com>... Jim,
I do have some experience, maybe this is the most complex project I
have dealt with, but we have enough time and resources to improve the
skills if required..
anyway the project is not so mission critical, even if a lot of data
are expected, and from a general point of view the factors are quite
calibrated
anyway we used range partitioning to handle the issue
best regards
jc
"Jim Kennedy" <ke************ *************** *@attbi.net> wrote in message news:<HgI%b.298 33$AL.531756@at tbi_s03>... "curwen" <jo**********@d espammed.com> wrote in message
news:cc******** *************** ***@posting.goo gle.com... > > jc
> I always find it strange that we see requests like this. Sort of along the > lines of "I am building this huge Oracle database and I don't know much
> about Oracle but..." Wouldn't one get people on the project who have some > knowledge and expertise on Oracle? It just strikes me as very odd.
> Jim
My experience in computer science is limited, I only have a First
Class Masters degree in Electronics Engineering, but I'm confident
with hard working and studying I can become 'good' as you
plus, I'll be much more professional answering in Usenet
thanks anyway
jc
I am sure you are a hard working good person. My comment has nothing to do
with your personality, but rather why people do this. (more the fault of
the manager from not getting experienced people for a task that needs an
experienced person) It is sort of like I've been hired to design a Boeing
747 and I have experience building paper airplanes. There is quite a gulf
there in needed experience.
Jim
"hrishy" <hr*****@yahoo. co.uk> wrote in message
news:4e******** *************** ***@posting.goo gle.com... Hi
Engineer from india here :-)..Well there are many different ways to
ask a question and depending on the way its asked you would get many
different responses :-).
Your problem can it be understood this way you want to better manage
archival data that keeps on growing but is huge and you need it
online.
Well one possible solution would be to dig into the oracle Docs and
look into partitioning.Wh ich is divide and conquer approcah for
managing data .Specifically look at range partitioning.
well if you want to be sure that usenet advice really works for you
you should take a hint and start doing test for yourself.You are smart
hmm and you will take every advice offered here and test it out.
And if you are too lazy well indian engineer here would welcome you
with open arms ..remember outsourcing is just a arms length away. :-))
regards
Hrishy
P.S:Please keep outsourcing and other bull shit away from this news
group folks.
I hate it this forum is meant for oracle and technical discussions.
I've seen a lot of Indian engineers do this very thing. I've also seen
non-Indian engineers do this type of thing. I don't think it is a
nationality or country of origin thing.
Jim
jo**********@de spammed.com (curwen) wrote in message
news:<cc******* *************** ****@posting.go ogle.com>... Jim,
I do have some experience, maybe this is the most complex project I
have dealt with, but we have enough time and resources to improve the
skills if required..
anyway the project is not so mission critical, even if a lot of data
are expected, and from a general point of view the factors are quite
calibrated
anyway we used range partitioning to handle the issue
best regards
jc
"Jim Kennedy" <ke************ *************** *@attbi.net> wrote in
message news:<HgI%b.298 33$AL.531756@at tbi_s03>... "curwen" <jo**********@d espammed.com> wrote in message
news:cc******** *************** ***@posting.goo gle.com...
> > > jc
> > I always find it strange that we see requests like this. Sort of
along the > > lines of "I am building this huge Oracle database and I don't know
much > > about Oracle but..." Wouldn't one get people on the project who
have some > > knowledge and expertise on Oracle? It just strikes me as very
odd. > > Jim
>
>
> My experience in computer science is limited, I only have a First
> Class Masters degree in Electronics Engineering, but I'm confident
> with hard working and studying I can become 'good' as you
>
> plus, I'll be much more professional answering in Usenet
>
> thanks anyway
> jc
I am sure you are a hard working good person. My comment has nothing
to do with your personality, but rather why people do this. (more the fault
of the manager from not getting experienced people for a task that needs
an experienced person) It is sort of like I've been hired to design a
Boeing 747 and I have experience building paper airplanes. There is quite a
gulf there in needed experience.
Jim
> Engineer from india here :-)..Well there are many different ways to ask a question and depending on the way its asked you would get many
different responses :-).
And if you are too lazy well indian engineer here would welcome you
with open arms ..remember outsourcing is just a arms length away. :-))
regards
Hrishy
P.S:Please keep outsourcing and other bull shit away from this news
group folks.
I hate it this forum is meant for oracle and technical discussions.
I find it very weird:
you just raise a plain technical question and so many etherogeneous
and *OT* topics come out
a simple answer like: 'try use partitioning'
would have been too complex, isn't
PS even silence is a good option sometimes This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Bart Goeman |
last post by:
Hi,
I have a question about how to put redundant information in data
structures, initialized at compile time. This is often necessary
for performance reasons and can't be done at run time (data
structures are read only)
Ideally one should be able to put the redundant
information there automatically so no mistakes are possible, but in a lot
of case I see no way how to do it.
|
by: Al Smith |
last post by:
I need help in implementing proper error handling.
I am trying to upload a file based on the sample code below. The code works
well except if the file selected is too big.
I do know about the maxRequestLength parameter of the <httpRuntime> section
and that works as expected. What I want is to enforce a max file size but
haven't been able to trap the error thrown when the file is too large and
that's where I could use some help.
|
by: Gustaf |
last post by:
I'm confused about structured error handling. The following piece of
code is a simplification of a class library I'm working on. It works,
and it does what I want, but I'm still not convinced that I have been
doing it right. I think I overdo it. Please have a look:
--
using System;
using System.IO;
|
by: athos |
last post by:
Hi guys,
Got a problem now :( please help...
now we got a project handling records saved in a table in a sql
2000(will upgraded to 2005 soon) server. every month around a million
records will be inserted.
now user raised a request, that is, once criterios are matched, the
project should do some backend handle, for example, if
|
by: Richard Collette |
last post by:
When attempting to debug a webservice, I get the error:
Cannot serialize member System.Exception.Data of type
System.Collections.IDictionary, because it implements IDictionary.
In reading about web services
(http://msdn2.microsoft.com/en-us/library/ds492xtk.aspx), the documentation
states that exceptions will automatically be handled as a fault element in
the message and then converted to a SoapException in the client.
| | |
by: Arno |
last post by:
reposted with the right microsoft managed newsgroup ID: Sorry for the
inconvinience
Hi,
I've written a class for client-socket connection, but I get a lot of times
the error message "Unable to read data from the transport connection" when
restart reading the stream with socket.BeginRead in the Sub
SocketIncomingMsg. I'm debugging now for weeks, but I can't detect where
it's
|
by: Earl |
last post by:
I have all of my data operations in a separate library, so I'm looking for
what might be termed "best practices" on a return type from those classes.
For example, let's say I send an update from the UI layer to a method in a
library class that calls the stored procedure. Best to return a boolean
indicating success/failure, return a string with the exception message, or
just return the entire exception?
|
by: Zorro |
last post by:
The simplicity of stack unraveling of C++ is not without defective
consequences. The following article points to C++ examples showing the
defects. An engineer aware of defects can avoid hard-to-find bugs.
http://distributed-software.blogspot.com/2007/01/c-exception-handling-is-defective.html
Regards,
zorabi@ZHMicro.com
http://www.zhmicro.com
http://distributed-software.blogspot.com
|
by: The Frog |
last post by:
Hi Everyone,
I have a small problem that doesnt seem to make any sense. I am using
Access 97, and have a query that selects data from a text field,
converts it to type Lng. This seems to work fine. When I try and apply
a criteria for filtering is get the data type mismatch rubbish that
Access likes to trow at you.
Here is the SQL:
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| | |
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| | |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |
