473,394 Members | 1,866 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,394 software developers and data experts.

can I get a list of all http sites in the world?

3
I've tried inurl:http but it takes forever to get even a bunch of sites right and I have to think of new keywords everytime to get the sites. Is there kind of a directory or a script i could use to filter the http from all the sites on the web. Or some sort of software?
Jun 7 '19 #1

✓ answered by gits

holy - this is probably something that always will take ages unless you are going to filter out according to inactivity etc. To get a rough idea of how many websites exist and to go on from (may be you can find more detailed and useful information from there on) you can have a first look here:

https://www.internetlivestats.com/to...r-of-websites/

3 1382
gits
5,390 Expert Mod 4TB
holy - this is probably something that always will take ages unless you are going to filter out according to inactivity etc. To get a rough idea of how many websites exist and to go on from (may be you can find more detailed and useful information from there on) you can have a first look here:

https://www.internetlivestats.com/to...r-of-websites/
Jun 7 '19 #2
4yush
3
thanks, gits I am sorry for my absurdity and wasting your time. One last thing?
If that wont work then Can I create a script which could filter out all the https results in a search result leaving only http results and repeat that process over and over and scrape all the website addresses using a web scraper plugin or something
Jun 7 '19 #3
gits
5,390 Expert Mod 4TB
well - so what exactly do you have and want to achieve? do you have a searchresult with urls that you want to filter for http/https or what exactly? basically its just a simple string comparison or using a regexp to find out all urls with http vs https from a list of strings (urls). but that wasnt my understanding of your question in the first place - where you wanted to do that with 'all http sites in the world'. if you have a search result already - where you just want to filter out the urls that start with http it should be pretty simple with a regexp. so long story short - in that case - yes you can create such script.
Jun 7 '19 #4

Sign in to post your reply or Sign up for a free account.

Similar topics

0
by: Mike | last post by:
Sites using thumbnail preview for world wide web file navigation and searching. Below are list of sites that are either researching or providing thumbnail preview images for online web...
9
by: LRW | last post by:
So I've been using CSS for a year now pretty much only as a replacement for FONT tags. I know you can do a LOT more with them, though. Like, eliminate the use for tables. Are there any good Web...
9
by: Henrik Goldman | last post by:
Hi, I would like to create a simplistic parser which goes through each .h file and finds each function prototype (or inline implementation) along with class names and member functions. ...
4
by: phcmi | last post by:
I have a PropertyGrid question. My task is to replace a legacy dialog box presentation with a modern one. The dialog itself allows the user to set configuration settings in our application, so...
8
by: Shraddha | last post by:
Can we access a node without traversing the list........ I mean if a list is having total 10 nodes..and I want to jump directly on 5th node without traversing first four nodes can I do that? The...
1
by: madlen | last post by:
I have a list of sites from where I want to scrap the data from.. I can pay you to make some scripts for me. I have already bought some scripts for yellowpages but I want to add some more categories...
19
by: Spiros Bousbouras | last post by:
Every time I've seen an example of a variable argument list function its functionality was to print formatted output. Does anyone have examples where the function is not some variation of printf ?
2
by: =?Utf-8?B?WWFua2VlIEltcGVyaWFsaXN0IERvZw==?= | last post by:
how can i disable a specific button in a list that is bound to an object (datasource)? I can, it seems disable a button that's added manuelly. Any help appreciated, thanks -- Share The...
10
freddieMaize
by: freddieMaize | last post by:
Hi there, I need to hit few list of sites from my application and find the response time of those site and sort them (which ever is opening first will sit at the top). I have done the same using...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.