I am trying to build a chatbot like Jarvis. I am using Convai API for that. However, when I ask for certain things that are new (i.e. information about an event that happened yesterday, which was not available for the API since it is pre trained on old dataset). So, I want to get information from Google if the chatbot doesn't know the answer.
However, if I search Google using Beautiful Soup and requests, the results are not accurate (e.g. When I ask, "when will the next episode of one piece will come out" the answer is "Episode 13") So please tell me what I can do to scrape the correct answer for my question. The following code is used for scraping the internet.
from bs4 import BeautifulSoup
import requests
def google_search(self, query):
url = f"https://www.google.com/search?q={query}"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")
search_results = soup.find_all("div", class_="BNeawe")
answer = [result.text for result in search_results]
return answer[0]