I'm accessing an endpoint that is responding with two objects almost at the same time with a content-type of 'text/event-stream'.
I'm struggling to access this data in Python.
I tried turning it into JSON with the .json() function, however that seems to not be working.
Does anyone have any ideas on how I can access the data from this response?
Here is the response in the network tab:
My Code:
import requests
request_session = requests.session()
initialResponse = request_session.post('https://transport.tamu.edu/busroutes.web/mapHub/negotiate?negotiateVersion=1').json()
connectionToken = initialResponse['connectionToken']
finalResponse = request_session.get(f'https://transport.tamu.edu/busroutes.web/mapHub?id={connectionToken}') # .json() not working
print(finalResponse) # 200 OK Response
I tried using the SSEClient library as such, but can only see an empty string:
import requests
from sseclient import SSEClient
import json
request_session = requests.session()
initialResponse = request_session.post('https://transport.tamu.edu/busroutes.web/mapHub/negotiate?negotiateVersion=1').json()
connectionToken = initialResponse['connectionToken']
sse = SSEClient(f'https://transport.tamu.edu/busroutes.web/mapHub?id={connectionToken}')
for event in sse:
if event.event == 'message':
try:
# Parse the event data as JSON
json_data = json.dumps(event.data)
print(json_data)
# Handle or process the JSON data as needed
# For example, you can access json_data['key'] to access specific fields
except json.JSONDecodeError as e:
print(f"Error parsing JSON: {e}")
else:
# Handle other types of events if needed
pass
For reference, I'm trying to scrape the network requests from this site: 'https://transport.tamu.edu/busroutes.web/Routes?r=01'
Thanks!

