Main thread slows down threads 2-3x times

390 views Asked by At

I have 2 files - first of them parser_thread_emulation.py that spawns threads that do what is inside parser_emulation.py file and main () has while infinite loop as well.

File parser_thread_emulation.py:

import threading

import parser_emulation
from parser_emulation import *

class threadParser (threading.Thread):
   def __init__(self, group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None):
      threading.Thread.__init__(self, group, target, name, args, kwargs, daemon=daemon)
      self._return = None
   def run(self):
      if self._target is not None:
          print ("Starting thread " + self.name + "\n")
          self._target(*self._args, **self._kwargs)
          print("Finishing thread " + self.name + "\n")

   def join(self):
      threading.Thread.join(self)


def main ():

    global resp

    #for loop that launches n threads (for this example 1 thread)
    for i in range(5):
        time.sleep(0.1)
        threadParserStart = threadParser(target=get_request, name=f"Thread-Parser {i + 1}")
        threadParserStart.start()

    #Do not execute main thread while loop when it's commented
    #time.sleep(2000)

    #While loop that constantly does something
    while (True):
        a = 1
        continue


if __name__ == '__main__':
    main()

File parser_emulation.py:

import httplib2
import time

def get_request ():

    global resp
    resp = ""

    iteration = 0
    start_time = time.time()

    #While loop for constant parsing, facebook as an example
    while (True):
        http = httplib2.Http()
        resp, page = http.request("https://facebook.com")
        iteration+=1
        if (iteration % 10 == 0):
            print(f"Number or iterations passed: {iteration}", f"\nTime passed: {time.time() - start_time}\n")

When I execute main () and it doesn't have while infinite loop (pause is uncommented) working time result is next:

Number or iterations passed: 10 
Time passed: 4.593774795532227

If I comment pause #time.sleep(2000) and launch while loop in main thread then time result is next:

Number or iterations passed: 10 
Time passed: 10.95308804512024

So it becomes more then 2 times slower. How to bypass it and what could be other faster solution ?

0

There are 0 answers