473,790 Members | 3,200 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Creating search engine in Python Good or Bad [in performance]

I am now creating IR engine in python which has its own database
implementation . For nearly 2GB files it works fast. Can i expect the
same speed when my database goes large. Else i have to chose other
language[c/c++] for the speed.......... ...
Please tell me the solution....... ............... ..........

Apr 19 '07 #1
2 1240
Clement wrote:
I am now creating IR engine in python which has its own database
implementation . For nearly 2GB files it works fast. Can i expect the
same speed when my database goes large. Else i have to chose other
language[c/c++] for the speed.......... ...
Please tell me the solution....... ............... ..........
Your speed will be good if your database design is good. We don't have
a crystal ball out here to see into your database design. It is also
not clear what the bottlenecks to "speed" would be for you. If it is
disk I/O or LAN bandwidth C won't help at all.

-Larry
Apr 19 '07 #2
That's right. It all depends on your design. When you do a select you
could brute force it to to return the data, but the speed comes from
elegant designing and programing skills. Using C++ or even Better C
would probably boost your database model. Try to sort the data in the
database so you can efficient use Hash-es
On Apr 19, 8:46 pm, Larry Bates <larry.ba...@we bsafe.comwrote:
Clement wrote:
I am now creating IR engine in python which has its own database
implementation . For nearly 2GB files it works fast. Can i expect the
same speed when my database goes large. Else i have to chose other
language[c/c++] for the speed.......... ...
Please tell me the solution....... ............... ..........

Your speed will be good if your database design is good. We don't have
a crystal ball out here to see into your database design. It is also
not clear what the bottlenecks to "speed" would be for you. If it is
disk I/O or LAN bandwidth C won't help at all.

-Larry

Apr 19 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2385
by: Auction software | last post by:
Free download full version , all products http://netauction8.url4life.com/ Groupawy --------------- Google Groups Email spider. The first email spider for google groups. Millions of valid and active emails in one easy location to collect. Spiderawy
2
1844
by: jlrodilla | last post by:
hi all, i´m doing a search engine using python for the spider and php to make a web for the search. The Database i have choosen is postgreSQL. Do you think it is a good choosen? Any suggestion? if anybody in interested in colaborate please send an e-mail to jlrodilla@gmail.com In a few weeks i will upload all the code to one domain and so i can be able to send you information about this project.
11
4235
by: Petre Huile | last post by:
I have designed a site for a client, but they have hired an internet marketing person to incrase their search engine ranking and traffic. He wants to put extra-large fonts on every page which will make the design looks a bit rediculous. He also said that the big text cannot be hidden. I am just trying to find a compromise. Here are the questions: (1) Is it true that a page with an <H1> tag and very big font size will make a search...
2
2477
by: Zambo via SQLMonster.com | last post by:
Hi! We have Sql Server 2000 in our server (NT 4). Our database have now about +350.000 rows with information of images. Table have lot of columns including information about image name, keywords, location, price, color mode etc. So our database don?t include the images itself, just a path to the location of every image. Keywords -field have data for example like this: cat,animal,pet,home,child with pet,child. Now our search use Full-Text...
10
1796
by: Harlin Seritt | last post by:
Hi, Is anyone aware of an available open-source/free search engine app (something similar to HTDig) written in Python that is out there? Googling has turned up nothing. Thought maybe I'd mine some of you guys' minds on this. thanks, Harlin Seritt
4
1703
by: corebump | last post by:
hi everybody, i planinng develop a search engine and i think using the python. Python performance is enough this project? Best Regards
0
2028
by: Auction software | last post by:
Free download full version , all products from Mewsoft dot com http://netauction8.url4life.com/ Groupawy --------------- Google Groups Email spider. The first email spider for google groups. Millions of valid and active emails in one easy location to collect. Spiderawy
8
2124
by: Bruno Rafael Moreira de Barros | last post by:
I have this framework I'm building in PHP, and it has Search Engine Friendly URLs, with site.com/controller/page/args... And on my View files, I have <?=$this->baseURL;?to print the base URL on the links (eg. <a href='<?=$this->baseURL;?>/controller/page/args'>Go somewhere</ a>. But on the CSS / JS files, how will I do it? I wonder, because on the View files, I can do <?=$this->baseURL;?>/css/site.css, and it will work. But images on the...
47
3451
by: Henning_Thornblad | last post by:
What can be the cause of the large difference between re.search and grep? This script takes about 5 min to run on my computer: #!/usr/bin/env python import re row="" for a in range(156000): row+="a"
0
9666
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9512
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10419
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10201
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
9987
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9023
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5424
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5552
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
3
2910
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.