473,507 Members | 2,388 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Search is too slow? Why?

mp
Hi,

MS Access DB, C#, VS, SQL

I have implemented search with SQL statements like follows:
SQLString = "SELECT ENGLISH FROM MyTable WHERE ENGLISH LIKE
'"+txtWordManipulation.Text+"%' ORDER BY ENGLISH";

end everything is desperately slow. DB is more than 150k words and I haven't
defined primary key because indexed option is set on Yes (Duplicates OK).

Do you have some suggestions or idea for different search?

Thanks
Nov 16 '05 #1
7 1560
First of all, I haven't used Access very much, so I'm not sure what the
limitations are. I would expect that doing a LIKE with the "%" on the right
would allow the index to be used and shouldn't be too slow. On the other
hand, 150K records sounds like a lot for an Access database.

You might want to figure out some sort of alternate indexing scheme. For
example, you could make a table that contains the first two letters of the
"ENGLISH" column with an index into MyTable for matching records. Depending
on how large the rows are in MyTable, scanning the index table may be much
faster.

Assuming MyTable look something like this:

MyTable
- int - MyTableID
- char(255) - ENGLISH
- other fields follow -

you could do:

MyIndexTable
- int - MyTableID
- char(2) - MyTableStart
the MyTableStart field would simply contain the first two characters of the
ENGLISH field from MyTable. You could do a select like:
string shortString = string.empty;
if (txtWordManipulation.Text.Length > 1)
{
shortString = txtWordManipulation.Text.Substring(0,2);
}
else
{
shortString = txtWordManipulation.Text[0].ToString();
}

SqlString = "SELECT ENGLISH FROM MyTable WHERE ENGLISH LIKE
'"+txtWordManipulation.Text+"%' and MyTableID in (SELECT MyTableID FROM
MyIndexTable Where MyTableStart = '" + shortString + "')";

I honestly don't know if this would be faster. You'd need to test it, but
the subquery should restrict the search to a much smaller subset of MyTable
and may result in a significant increase in speed.

There are a number of variables that can affect this, however.

If the rows of MyTable are not particularly large, then this technique
doesn't help you much. Also, this technique will slow down inserts, updates,
and deletes as you'll have to update two tables.

Also, it's likely the records in MyTable are spread out across the table,
requiring most of the database to be read in anyway.

Again, I don't know Access very well, so I don't know how it handles the
data internally, but it may be possible that if you generate the table by
inserting the records in alphabetical order (such that words that have
similar starting strings are close together), that it may be faster as well,
but this depends on whether access keeps the records physically in the order
that they're inserted. I don't know that it does this.

Anyway, hope this gives you some possibilities.

Pete

"mp" <my***@volja.net> wrote in message
news:up**************@TK2MSFTNGP12.phx.gbl...
Hi,

MS Access DB, C#, VS, SQL

I have implemented search with SQL statements like follows:
SQLString = "SELECT ENGLISH FROM MyTable WHERE ENGLISH LIKE
'"+txtWordManipulation.Text+"%' ORDER BY ENGLISH";

end everything is desperately slow. DB is more than 150k words and I haven't defined primary key because indexed option is set on Yes (Duplicates OK).

Do you have some suggestions or idea for different search?

Thanks

Nov 16 '05 #2
Can you wquantify your complaints?

*What is slow in your eyes?
*If the access database on a network share?
*What is the content of txtWordManipulation.Text
*What is the table structure, plus all defined indices.

Btw:
*You do know you allow me to delete your complete database and run any SQL
statement I want with this stupid code of yours? SQL Injection attack, plain
and simple.

--
Regards

Thomas Tomiczek
THONA Software & Consulting Ltd.
(Microsoft MVP C#/.NET)
(CTO PowerNodes Ltd.)
---

Still waiting for ObjectSpaces? Try the EntityBroker today - more versatile,
more powerfull.
And something in use NOW. for the projects you have to deliver - NOW.
"mp" <my***@volja.net> wrote in message
news:up**************@TK2MSFTNGP12.phx.gbl...
Hi,

MS Access DB, C#, VS, SQL

I have implemented search with SQL statements like follows:
SQLString = "SELECT ENGLISH FROM MyTable WHERE ENGLISH LIKE
'"+txtWordManipulation.Text+"%' ORDER BY ENGLISH";

end everything is desperately slow. DB is more than 150k words and I haven't defined primary key because indexed option is set on Yes (Duplicates OK).

Do you have some suggestions or idea for different search?

Thanks

Nov 16 '05 #3
> Btw:
*You do know you allow me to delete your complete database and run any SQL
statement I want with this stupid code of yours? SQL Injection attack, plain and simple.


You must be one of the most exceptional programmers in the world to be so
confident that your own code is completely without error. I don't think it's
any help to call someone's code "stupid". Nothing wrong in pointing out
mistakes, but some of us come here to help and learn. Not be insulted.
Nov 16 '05 #4
<pd******@hotmail.com> wrote in message
news:26******************************@news.meganet news.com...
Btw:
*You do know you allow me to delete your complete database and run any SQL statement I want with this stupid code of yours? SQL Injection attack, plain
and simple.


You must be one of the most exceptional programmers in the world to be so
confident that your own code is completely without error. I don't think

it's any help to call someone's code "stupid". Nothing wrong in pointing out
mistakes, but some of us come here to help and learn. Not be insulted.


It is no insult.

This is a beginner 101 error.

Basically, if you do not know what a SQL Injection attack is, you should not
be allowed near code that runs against a SQL database.

This is like a doctor not knowing that people may die when loposing too much
blood.

It is BASIC.

Everyone does make errors, but a SQL Injection attack possibility by SUCH
code deserves to be called stupid code.

--
Regards

Thomas Tomiczek
THONA Software & Consulting Ltd.
(Microsoft MVP C#/.NET)
(CTO PowerNodes Ltd.)
---

Still waiting for ObjectSpaces? Try the EntityBroker today - more versatile,
more powerfull.
And something in use NOW. for the projects you have to deliver - NOW.
Nov 16 '05 #5
mp
Thanks for your opinion about my code. You are right about possibilities of
attack, but decent men should suggest that on different way. Your approach
is not professional, especially for person from consultants company.

I'll avoid possibility of this attack later with parameterized queries and
user inputs validations.

Anyway, I am not able to see connection between possibilities of SQL
Injection attack and my question???

I am sorry but I never heard for you. I would like to believe, you a
brilliant mind and extraordinary programmer.

I am sure you have solution for this problem. Do you? May be not, may be you
are one ordinary man and average programmer. But do not worry, here is a lot
of folks to help you.

Please keep in sight and do not forget, this place is home of beginners and
extraordinary professionals who helps them.

Bye,
Miro

"Thomas Tomiczek [MVP]" <t.********@thona-consulting.com> wrote in message
news:O5**************@TK2MSFTNGP12.phx.gbl...
Can you wquantify your complaints?

*What is slow in your eyes?
*If the access database on a network share?
*What is the content of txtWordManipulation.Text
*What is the table structure, plus all defined indices.

Btw:
*You do know you allow me to delete your complete database and run any SQL
statement I want with this stupid code of yours? SQL Injection attack, plain and simple.

--
Regards

Thomas Tomiczek
THONA Software & Consulting Ltd.
(Microsoft MVP C#/.NET)
(CTO PowerNodes Ltd.)
---

Still waiting for ObjectSpaces? Try the EntityBroker today - more versatile, more powerfull.
And something in use NOW. for the projects you have to deliver - NOW.
"mp" <my***@volja.net> wrote in message
news:up**************@TK2MSFTNGP12.phx.gbl...
Hi,

MS Access DB, C#, VS, SQL

I have implemented search with SQL statements like follows:
SQLString = "SELECT ENGLISH FROM MyTable WHERE ENGLISH LIKE
'"+txtWordManipulation.Text+"%' ORDER BY ENGLISH";

end everything is desperately slow. DB is more than 150k words and I

haven't
defined primary key because indexed option is set on Yes (Duplicates OK).
Do you have some suggestions or idea for different search?

Thanks


Nov 16 '05 #6
"mp" <my***@volja.net> wrote in message
news:OB**************@TK2MSFTNGP09.phx.gbl...
Thanks for your opinion about my code. You are right about possibilities of attack, but decent men should suggest that on different way. Your approach
is not professional, especially for person from consultants company.
Well, so far I have made it part of my life not to lie. I see a thief, I
call him a tihief. I see idiotic code, I call it idiotic code.

I'll avoid possibility of this attack later with parameterized queries and
user inputs validations.
Technically all you need to do is your homework with especial char handling.
If you are doing some little replaces on the iunput string, you are fine.
Anyway, I am not able to see connection between possibilities of SQL
Injection attack and my question???
This is totally your problem, because...

....in my answer I asked some questions about your code, and also made a
comment about your openness for SQL Injection.

You cared to ignore my answers and focus on the comment of the code.

So, what do you expect?

You did not provide enough information for sensible help, and have not
answered my questions yet. Without he answers I do not feel comfortable to
answer.
I am sorry but I never heard for you. I would like to believe, you a
brilliant mind and extraordinary programmer.

I am sure you have solution for this problem. Do you? May be not, may be you are one ordinary man and average programmer. But do not worry, here is a lot of folks to help you.
Spoken like someone who has no clue about SQL Injection attacks, and as
someone who - can not read. Like my answer and my questions to your problem.

I surely have a solution. IOnteresting point is, though, that your problem
technically SHOULD be a non-issue. This is why I asked questions.

That you tend to ignore them is basically one exact thing: your fault. Would
you have answered them, you would have been helped by now.
Please keep in sight and do not forget, this place is home of beginners and extraordinary professionals who helps them.
Well, they should start learning to read before trying to program.

Means: when you come asking for help, and are asked to provide more
information, you should provide it if you expect a sensible answer.
--
Regards

Thomas Tomiczek
THONA Software & Consulting Ltd.
(Microsoft MVP C#/.NET)
(CTO PowerNodes Ltd.)
---

Still waiting for ObjectSpaces? Try the EntityBroker today - more versatile,
more powerfull.
And something in use NOW. for the projects you have to deliver - NOW.

Nov 16 '05 #7
On Sun, 13 Jun 2004 21:48:54 +0200, "Thomas Tomiczek [MVP]"
<t.********@thona-consulting.com> wrote:
<pd******@hotmail.com> wrote in message
news:26******************************@news.megane tnews.com...
> Btw:
> *You do know you allow me to delete your complete database and run anySQL > statement I want with this stupid code of yours? SQL Injection attack,

plain
> and simple.


You must be one of the most exceptional programmers in the world to be so
confident that your own code is completely without error. I don't think

it's
any help to call someone's code "stupid". Nothing wrong in pointing out
mistakes, but some of us come here to help and learn. Not be insulted.


It is no insult.

This is a beginner 101 error.

Basically, if you do not know what a SQL Injection attack is, you should not
be allowed near code that runs against a SQL database.

This is like a doctor not knowing that people may die when loposing too much
blood.

It is BASIC.

Everyone does make errors, but a SQL Injection attack possibility by SUCH
code deserves to be called stupid code.


I don't like to but in here, and i try to avoid it in proffessional
newsgroups such as this.

I thought the origonal poster asked a fairly easy question, which had
no mention to SQL injection at all. Now fair enough that this could be
a problem, but you could use a little tact. I know of seasoned DB
programmers who had never heard of SQL injection until a recent
ADO.net course, so it's not just a beginners problem. Add to that, the
code the OP is providing is a snippet, you do not know what is going
on behind the scenes.

PS, for those other programmers, myself being a hardware engineer and
frequently visiting h/w NG's this thread is very tame, i can assure
you! ;) I do enjoy the professional behaviour on MS programming NG's,
it's nice to see.

Nov 16 '05 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
1691
by: RiceGuy | last post by:
Hi! I'm looking for ideas on what would the best approach to design a search system for a RSS feeds. I will have some 50 RSS feeds (all RSS 2.0 compliant) stored locally on the web server. Now I'm...
2
2465
by: Zambo via SQLMonster.com | last post by:
Hi! We have Sql Server 2000 in our server (NT 4). Our database have now about +350.000 rows with information of images. Table have lot of columns including information about image name, keywords,...
83
5843
by: D. Dante Lorenso | last post by:
Trying to use the 'search' in the docs section of PostgreSQL.org is extremely SLOW. Considering this is a website for a database and databases are supposed to be good for indexing content, I'd...
7
2601
by: spike | last post by:
Im writing a program to search for a string in a binary file. And it works. The problem is: It is sooo slow! how can i make it faster? It takes 27 seconds just to search a 5 meg file. I guess it...
12
6405
by: Vjay77 | last post by:
Hi, I haven't posted any problem in quite a while now, but I came to the point that I really need to ask for help. I need to create an application which will search through .txt log file and...
4
1541
by: neoform | last post by:
I was wondering what would be the best search application for indexing/ searching content? Fulltext in MySQL is too slow once you get a db over 1GB. I played around with Zend's version of Lucene,...
0
7223
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
1
7031
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
7485
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
5623
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
1
5042
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
4702
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
0
3191
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The...
0
3179
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
0
1542
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.