By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
458,028 Members | 1,309 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 458,028 IT Pros & Developers. It's quick & easy.

Scrape Google search results of specific date range

P: 15
Hi,
I am trying to build a system that allows me to pull Google search results of specific date range to my local machine. I tried the following code which generates the result as shown in image shown in the following link. Doesn't show date range.

Expand|Select|Wrap|Line Numbers
  1. import requests, bs4
  2.  
  3. res = requests.get('https://www.google.com/search?q=nepal+OR+नेपाल&tbs=cdr:1,cd_min:4/1/2020,cd_max:4/8/2020')
  4. res.raise_for_status()
  5.  
  6. soup = bs4.BeautifulSoup(res.text, 'html.parser')
  7. linkElements = soup.select('div')
  8.  
  9.  
  10. open('output.html', 'w').write(res.text)
  11.  
It is showing a normal search result for 'nepal OR नेपाल' but I need the search results for a date range lets say April 1, 2020 to April 8, 2020 as shown in the image in the link below.
Result with date range

With some research, I came to know that this can be done by the use of APIs such as SerpAPI but that will cost me 50$ per month.

Is there any other way I could do this?
Apr 8 '20 #1
Share this Question
Share on Google+
4 Replies


Rabbit
Expert Mod 10K+
P: 12,427
Google itself provides an API that you can use for free 100 times a day
Apr 8 '20 #2

P: 15
Hi,
Thank you for your reply. Could you please specify the actual name of the API that I can use to do this?
Apr 9 '20 #3

Rabbit
Expert Mod 10K+
P: 12,427
It's on Google's developer site, here's a link https://developers.google.com/custom-search/v1/overview
Apr 9 '20 #4

P: 15
Tried that. It is not quite what I am looking for.
Apr 11 '20 #5

Post your reply

Sign in to post your reply or Sign up for a free account.