473,668 Members | 2,265 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

can I get a list of all http sites in the world?

3 New Member
I've tried inurl:http but it takes forever to get even a bunch of sites right and I have to think of new keywords everytime to get the sites. Is there kind of a directory or a script i could use to filter the http from all the sites on the web. Or some sort of software?
Jun 7 '19 #1
3 1389
gits
5,390 Recognized Expert Moderator Expert
holy - this is probably something that always will take ages unless you are going to filter out according to inactivity etc. To get a rough idea of how many websites exist and to go on from (may be you can find more detailed and useful information from there on) you can have a first look here:

https://www.internetlivestats.com/to...r-of-websites/
Jun 7 '19 #2
4yush
3 New Member
thanks, gits I am sorry for my absurdity and wasting your time. One last thing?
If that wont work then Can I create a script which could filter out all the https results in a search result leaving only http results and repeat that process over and over and scrape all the website addresses using a web scraper plugin or something
Jun 7 '19 #3
gits
5,390 Recognized Expert Moderator Expert
well - so what exactly do you have and want to achieve? do you have a searchresult with urls that you want to filter for http/https or what exactly? basically its just a simple string comparison or using a regexp to find out all urls with http vs https from a list of strings (urls). but that wasnt my understanding of your question in the first place - where you wanted to do that with 'all http sites in the world'. if you have a search result already - where you just want to filter out the urls that start with http it should be pretty simple with a regexp. so long story short - in that case - yes you can create such script.
Jun 7 '19 #4

Sign in to post your reply or Sign up for a free account.

Similar topics

0
5606
by: Mike | last post by:
Sites using thumbnail preview for world wide web file navigation and searching. Below are list of sites that are either researching or providing thumbnail preview images for online web documents. Thumbnail previews are useful for web site navigation particularly in search engines and directories such as Google, Altavista and Yahoo. The preview images provide a portion of the content of the electronic file to aid in navigation.
9
775
by: LRW | last post by:
So I've been using CSS for a year now pretty much only as a replacement for FONT tags. I know you can do a LOT more with them, though. Like, eliminate the use for tables. Are there any good Web sites for the newbie to semi-beginner on using CSS beyond just font attributes? Thanks for any suggestions!! Liam druid-at-celticbear-dot-com
9
3606
by: Henrik Goldman | last post by:
Hi, I would like to create a simplistic parser which goes through each .h file and finds each function prototype (or inline implementation) along with class names and member functions. Examples: test.h:
4
18817
by: phcmi | last post by:
I have a PropertyGrid question. My task is to replace a legacy dialog box presentation with a modern one. The dialog itself allows the user to set configuration settings in our application, so it seems to me that a PropertyGrid is a perfect modern replacement. I won't bore you with the details with all the controls on the dialog since I'm only concerned with one issue. The legacy dialog contains a drop down list control that is...
8
416
by: Shraddha | last post by:
Can we access a node without traversing the list........ I mean if a list is having total 10 nodes..and I want to jump directly on 5th node without traversing first four nodes can I do that? The list is singly list... I tried it using macros...I just thought of it....But I can not code it.. This is what I thought..
1
1666
by: madlen | last post by:
I have a list of sites from where I want to scrap the data from.. I can pay you to make some scripts for me. I have already bought some scripts for yellowpages but I want to add some more categories to my database and I don't know to modify the script that I have from the coders. they are made in python (i can modify them only to change the category from where i want the data to be extracted, but if I change the website they don't work and I...
19
2123
by: Spiros Bousbouras | last post by:
Every time I've seen an example of a variable argument list function its functionality was to print formatted output. Does anyone have examples where the function is not some variation of printf ?
2
1204
by: =?Utf-8?B?WWFua2VlIEltcGVyaWFsaXN0IERvZw==?= | last post by:
how can i disable a specific button in a list that is bound to an object (datasource)? I can, it seems disable a button that's added manuelly. Any help appreciated, thanks -- Share The Knowledge. I need all the help I can get and so do you!
10
5233
freddieMaize
by: freddieMaize | last post by:
Hi there, I need to hit few list of sites from my application and find the response time of those site and sort them (which ever is opening first will sit at the top). I have done the same using java (server side). Now my requirement is do in the client side (AJAX possibly). I’m having two challenges. Below is what I have done so far, <script> function call() { var d = new Date(); var xmlHttp;
0
8374
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8890
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8791
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8653
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7398
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5677
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4202
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4373
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
2018
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.