Multithreading in Python

Often we build applications which might require several tasks to run simultaneously within the same application. This is where the concept of multithreading in python comes into play. This post provides a comprehensive explanation of using Multithreading in Python aka Threading in Python.

Introduction

Multithreading in Python or Threading in python is a concept by which mutliple threads are launched in the same process to achieve parallelism and multitasking within the same application. Executing different threads are equivalent to executing different programs or different functions within the same process.

This post will cover how threading in python works with the presence of GIL or Global Interpreter Lock. Also, it will illustrate how a pool of threads in python can be created using a python threadpoolexecutor or python threadpool.

Table of Contents

• What is Multithreading or Threading
• When to use Multithreading in Python
• Threading Example in Python
• Global Interpreter Lock or GIL in Python
  • How does threading work in Python?
• What is Join in Threading
• Python Multithreading - Thread Pool
  • Using ThreadPoolExecutor in Python
  • Using ThreadPool in Python
• Synchronization between threads - Thread Lock
• When to not use Python Multithreading
• Conclusion

What is Multithreading(Threading) ?

Python Multithreading can be simply understood as executing multiple threads simultaneously within the same process. These threads share the same memory space as the process.
For example, a GUI such as Pycharm or a Jupyter Notebook keeps autosaving your code as and when you make changes, which clearly illustrates multiple tasks being performed within the same process.

Python Threading is generally preferred for lightweight processes since multiple tasks are run within the same process and uses the memory allocated for that process (i.e.) uses the memory space of that process.

Multithreading in python shouldn’t be confused with multiprocessing because multiprocessing is where two or more processes run in a single application without having a shared state within them because they run as different independent processes.

To read more about multiprocessing in python, please click here

When to use Multithreading in Python ?

Multithreading in python can be used for multiple use-cases, some of which are mentioned below,

• Threads are most suitable for lightweight tasks.
• When you have multiple tasks which are more I/O bound.
• Creating a lag free application with a backend database connection.
• When you want to make multiple calls to an API.
• When you want to download/upload data from/to a server.
• When you have objects being shared across the same application.

Important Note: It is highly recommended to use multithreading in python for I/O bound tasks alone due to the presence of GIL in CPython which is explained in detail below.

Threading Example in Python

To create a thread in python, you simply call the threading.Thread method as shown below,

Below example illustrates how to create a thread in python and how a function can be called inside a thread,

import threading

def new_function(a, b, kw2=10, kw1=None)
    print("Hello")

# Create a thread

new_thread = threading.Thread(target=function_name, args=(arg1, arg2), kwargs={'kw1': 1, 'kw2': '2'})

To start the above thread, run the below command,

new_thread.start()

Now that we have understood how to create and start a thread, let’s go ahead with another simple example of an application where you connect to different URLs using the requests module by launching 2 threads in python.

Note: This is just a just a simple example for the purpose of demonstrating threading in python.

import threading
import requests

def connect_to_urls(urls):
    for url in urls:
        r = requests.get(url)
        print(f"{url} - Status: {r.status_code}")

if __name__ == '__main__':
    
    url_list = ['https://google.com', 'https://youtube.com', 'https://wikipedia.com', 'https://medium.com']
    print(f"URL List: {url_list}")
    
    t1 = threading.Thread(target=connect_to_urls, args=(url_list[:2],)) # Create Thread 1
    t2 = threading.Thread(target=connect_to_urls, args=(url_list[2:],)) # Create Thread 2

    t1.start()  # Thread 1 starts here
    t2.start()  # Thread 2 starts here

URL List: ['https://google.com', 'https://youtube.com', 'https://wikipedia.com', 'https://medium.com']
https://wikipedia.com - Status: 200
https://google.com - Status: 200
https://medium.com - Status: 200
https://youtube.com - Status: 200

Global Interpreter Lock or GIL in Python

Before we get started with the actual threading modules in python, let’s first understand what GIL in python is and how it plays a role in python multithreading.

Global Interpreter Lock, otherwise known as GIL in Python is a mutex (or lock) which ensures only a single thread executes Python’s bytecode at any instant. It ensures thread safety to python’s objects and enables a single threaded access to the python’s interpreter.

GIL is an implementation of CPython, because of which true multithreading cannot be achieved in Python. Hence, if CPU bounds tasks are multi-threaded it can lead to increased execution times, which is why it is recommended to use multithreading in python for I/O bound tasks alone.

When a thread is running, it gets hold of the GIL while performing it’s task(s). This GIL is released during I/O operations which involve some waiting time such as read, write, send, download etc.

How does threading work in Python

CPython has the Global Interpreter Lock (GIL), which hinders multithreading in Python because at any moment only a single thread can acquire the GIL to work on Python’s bytecode. So now the question arises as to how concurrency is achieved in Python via threads.

Context switching takes place between the threads to enable some sort of concurrency. For example, imagine a requirement wherein you would have to run 10 different queries in a database independently. Each SQL query would take some time to complete it’s execution, so this wait time (I/O) is when thread switching takes place to enable concurrency.

To understand how threading in python works, in more detail, we have to take into consideration the type of tasks being run, namely, I/O Bound Tasks and CPU Bound Tasks

• I/O Bound Tasks

I/O bound tasks are tasks which involve some waiting time or delay for the operations to complete. Some examples of such tasks include, reading or writing data to a database/disk, downloading/uploading data etc.

For such tasks, whenever there is a wait time involved, GIL is released by the current thread and the next thread on queue will acquire the GIL and begin it’s execution. Let’s understand this better with an example.

Below is a pictorial representation of how threading works in python.

For our example, let’s consider 2 threads and a function which includes a 1 second sleep.

import threading
import time
import sys

def display_time(thread):
    """
    Display the time after a 1 second delay
    """
    print(f"Thread {thread} - Beginning to sleep at {time.time()}")
    time.sleep(1)
    print(f"Thread {thread} - Sleep Complete at {time.time()}")
    

if __name__ == '__main__':
    
    t1 = threading.Thread(target=display_time, args=[1])
    t2 = threading.Thread(target=display_time, args=[2])
    start = time.time()
    t1.start()
    t2.start()

    t1.join()
    t2.join()
    
    print(f"We are at the end of the program. Time Taken: {time.time() - start}")

Thread 1 - Beginning to sleep at 1590616707.353773
Thread 2 - Beginning to sleep at 1590616707.3544252
Thread 1 - Sleep Complete at 1590616708.354223
Thread 2 - Sleep Complete at 1590616708.359642
We are at the end of the program. Time Taken: 1.006382942199707

• CPU Bound Tasks

A task is CPU bound if the time it takes for its completion depends on the speed of the CPU. Some examples of CPU bound tasks include number crunching tasks, mathematical operations, calculations etc.

For such tasks, since there is no wait time involved here, thread switching takes place every 0.005 seconds which is determined by the sys.getswitchinterval. This can also be modified manually using sys.setswitchinterval. You can read more about it here.

Note: This type of thread switching after a specific time interval is defined from Python 3.2 onwards.

Let’s understand this better with an example. Here we will run a function which will decrement the value of counter until it reaches zero. We will be running the same function once again after increasing thread switch interval and see the results.

Important Note: It is not recommended to use multithreading in python for CPU bound applications as it might lead to performance degradations.

import threading
import time
import sys


def decrement(t):
    """
    Increment val 100000 times
    """
    start = time.time()
    print(f"Thread {t} in decrement at {start}")
    counter = 100000000
    while counter > 0:
        counter -= 1
    end = time.time()
    print(f"Thread {t} ended at {end}, Time Taken: {end - start}")  
    

def run_threads():
    t1 = threading.Thread(target=decrement, args=(1,))
    t2 = threading.Thread(target=decrement, args=(2,))
    
    start = time.time()
    t1.start()
    t2.start()
    
    t1.join()
    t2.join()

    print(f"We are at the end of the program. Total Time Taken: {time.time() - start}")

    
if __name__ == '__main__':
    print(f"Switch Interval is {sys.getswitchinterval()} seconds")
    run_threads()
    
    sys.setswitchinterval(10)
    print(f"\nSwitch Interval set to {sys.getswitchinterval()} seconds")

    run_threads()
    sys.setswitchinterval(0.005) # Setting it back to it's original value

Switch Interval is 0.005 seconds
Thread 1 in decrement at 1590616384.611721
Thread 2 in decrement at 1590616384.617289
Thread 2 ended at 1590616396.703609, Time Taken: 12.086319923400879
Thread 1 ended at 1590616396.8213172, Time Taken: 12.209596157073975
We are at the end of the program. Total Time Taken: 12.210150241851807

Switch Interval set to 10.0 seconds
Thread 1 in decrement at 1590616396.822057
Thread 1 ended at 1590616402.9130921, Time Taken: 6.0910351276397705
Thread 2 in decrement at 1590616402.913912
Thread 2 ended at 1590616408.976337, Time Taken: 6.062424898147583
We are at the end of the program. Total Time Taken: 12.155883073806763

What is join in Threading ?

Before we define what join in threading is, let’s analyse our problem statement.

There is an application running with 2 threads in parallel. Thread 1 completes 30 seconds faster than thread 2.

So, upon completion of the first thread, would the program exit or wait for 2nd thread to complete ?

Well, if you guessed the program would terminate, then you are right. This is where the magic of join method of a thread comes into play.

Join basically makes the program to wait for the thread to finish. So, an additional join after starting the threads would make the application wait to successfully complete all the threads in it.

Let’s make more sense of join with the below 2 examples,

Example 1 : Threading in python without Join

Let’s create a program to display the time thrice with a 0.5 second delay and have some code after the threads.

import threading
import time
import sys


def display_time(val):
    """
    Display the time 3 times 
    with a 0.5 second delay
    """
    for i in range(3):
        time.sleep(0.5)
        print("Process:{0} Time is {1}".format(val, time.time()))


if __name__ == '__main__':
    
    t1 = threading.Thread(target=display_time, args=(1,))
    t2 = threading.Thread(target=display_time, args=(2,))

    t1.start()
    t2.start()
    
    print("Threading Complete. We are at the end of the program.")

Threading Complete. We are at the end of the program.
Process:1 Time is 1543436647.7216341
Process:2 Time is 1543436647.722194
Process:1 Time is 1543436648.2265742
Process:2 Time is 1543436648.227299
Process:1 Time is 1543436648.729373
Process:2 Time is 1543436648.731555

It is evident from the above example that even before the 2 threads complete their execution, the line of code present after starting the 2 threads is being executed.

So, how do you wait for the threads to complete before you continue with the execution of the rest of the program ?

This is where join comes in as the perfect solution. Now let’s analyze the same example with join.

Here the program waits for the threads to complete before arriving to the end.

Example 2 : Threading in python with Join

import threading
import time
import sys


def display_time(val):
    """
    Display the time 3 times 
    with a 0.5 second dh elay
    """
    for i in range(3):
        time.sleep(0.5)
        print("Process:{0} Time is {1}".format(val, time.time()))


if __name__ == '__main__':
    
    t1 = threading.Thread(target=display_time, args=(1,))
    t2 = threading.Thread(target=display_time, args=(2,))

    t1.start()
    t2.start()
    
    t1.join()
    t2.join()
    
    print("Threading Complete. We are at the end of the program.")

Process:1 Time is 1543436975.869845
Process:2 Time is 1543436975.8704278
Process:2 Time is 1543436976.37433
Process:1 Time is 1543436976.37479
Process:2 Time is 1543436976.87863
Process:1 Time is 1543436976.878934
Threading Complete. We are at the end of the program.

Python Multithreading - Thread Pool

You can also start a pool of threads in python to run your tasks concurrently.

This can be achieved by using the ThreadPoolExecutor in python which is part of concurrent.futures or by using ThreadPool which is part of the multiprocessing module.

Below are 2 examples which illustrate the usage of both.

Example 1 - Using ThreadPoolExecutor in Python

We can import the ThreadPoolExecutor class from the concurrent.futures module to start a pool of worker threads.

For our example, we will download a random list of images using the requests module in Python.

Next we will initialize a ThreadPoolExecutor to start a pool of threads. The number of threads can be specified using the max_workers argument. If unspecified it will take the default value depending upon the version. In Python version 3.8, the default value of max_workers is min(32, os.cpu_count() + 4). For our example, we will use 5 threads.

In addition to this, we will also be comparing the time taken by perfoming a sequential execution of the same.

import time
from concurrent.futures import ThreadPoolExecutor
import requests


def save_image(url):
    """
    Method to download an image and save it as a local image file
    """
    r = requests.get(url)
    file_name = url.split("/")[-1]
    with open(file_name, 'wb') as f:
        f.write(r.content)
    print(f"{file_name} - Downloaded")
    return file_name

image_url_list = ['https://cdn.pixabay.com/photo/2020/05/20/06/47/rome-5195046__480.jpg', 
                  'https://cdn.pixabay.com/photo/2020/05/15/18/10/forth-bridge-5174535__480.jpg',
                  'https://cdn.pixabay.com/photo/2020/05/18/22/17/travel-5188598__480.jpg',
                  'https://cdn.pixabay.com/photo/2020/05/11/13/40/masonry-5158303__480.jpg',
                  'https://cdn.pixabay.com/photo/2020/05/17/12/56/mykonos-5181484__480.jpg',
                  'https://cdn.pixabay.com/photo/2020/02/11/07/55/robots-4838671__480.png',
                  'https://cdn.pixabay.com/photo/2020/05/12/16/43/mallard-5163882__480.jpg',
                  'https://cdn.pixabay.com/photo/2020/04/27/12/02/strawberries-5099527__480.jpg',
                  'https://cdn.pixabay.com/photo/2020/05/04/11/19/smile-5128742__480.jpg',
                  'https://cdn.pixabay.com/photo/2020/05/17/07/00/butterfly-5180349__480.jpg'
                 ]

start = time.time()

for img_url in image_url_list:
    save_image(img_url)
# _ = list(map(save_image, image_url_list))   # Alternative to above 2 statements

print(f"\nSequential Execution, Time Taken: {time.time() - start}\n")

start = time.time()

with ThreadPoolExecutor(max_workers=5) as executors:
    executors.map(save_image, image_url_list)  # Each item from the list is passed as arg to the function

print(f"\nParallel Execution, Time Taken: {time.time() - start}")

rome-5195046__480.jpg - Downloaded
forth-bridge-5174535__480.jpg - Downloaded
travel-5188598__480.jpg - Downloaded
masonry-5158303__480.jpg - Downloaded
mykonos-5181484__480.jpg - Downloaded
robots-4838671__480.png - Downloaded
mallard-5163882__480.jpg - Downloaded
strawberries-5099527__480.jpg - Downloaded
smile-5128742__480.jpg - Downloaded
butterfly-5180349__480.jpg - Downloaded

Sequential Execution, Time Taken: 1.1698040962219238

forth-bridge-5174535__480.jpg - Downloaded
mykonos-5181484__480.jpg - Downloaded
masonry-5158303__480.jpg - Downloaded
travel-5188598__480.jpg - Downloaded
rome-5195046__480.jpg - Downloaded
robots-4838671__480.png - Downloaded
mallard-5163882__480.jpg - Downloaded
strawberries-5099527__480.jpg - Downloaded
butterfly-5180349__480.jpg - Downloaded
smile-5128742__480.jpg - Downloaded

Parallel Execution, Time Taken: 0.2731809616088867

It is clearly evident from the above results that by utilizing threading in python, the overall execution time was reduced by a considerable amount.

Example 2 - Using ThreadPool in Python

The following example demonstrates how to use ThreadPool in Python to start a pool of threads.

Once we import this from the multiprocessing module, we will be using it’s map method to map each image url to the function and download them in parallel.

This is very similar to the Pool class and its methods are more or less the same. To read more about it, please click here.

import time
from multiprocessing.pool import ThreadPool

start = time.time()

threads = ThreadPool(5) # Initialize the desired number of threads
threads.map(save_image, image_url_list)

print(f"\nParallel Execution 2, Time Taken: {time.time() - start}")

travel-5188598__480.jpg - Downloaded
mykonos-5181484__480.jpg - Downloaded
rome-5195046__480.jpg - Downloaded
masonry-5158303__480.jpg - Downloaded
forth-bridge-5174535__480.jpg - Downloaded
robots-4838671__480.png - Downloaded
butterfly-5180349__480.jpg - Downloaded
strawberries-5099527__480.jpg - Downloaded
smile-5128742__480.jpg - Downloaded
mallard-5163882__480.jpg - Downloaded

Parallel Execution 2, Time Taken: 0.28809595108032227

Synchronization between threads - Thread Lock

Thread lock in Python helps establish a synchronization between threads and avoid race conditions. For example, what if you wish to share data among the running threads. This is the most useful part of the threading module showing how data can be shared across 2 or more threads in a synchronized way.

What happens if two or more threads try to make changes to a shared object at the same time ?
This would result in unexpected and asynchronous results. Thread locks help to combat this issue.

Thread lock in python is designed in such a way that at any instant only one thread can make changes to a shared object.

This locking mechanism ensures that a clean synchronization is established between the threads thereby avoiding unexpected results due to this simultaneous execution.

Practical Use Case: For example, sharing objects would be very useful in a case where there is a frontend UI to display a table’s data and this table’s data is manipulated from 2 data sources being refreshed periodically for every 5 minutes. So, if there’s a delay in any of these 2 data refreshes, and if both threads try to manipulate the same object at the same time, it might lead to inconsistent results.

Example 1 : Threading without Lock

Let’s create a scenario wherein a race condition is created, yielding inconsistent results.

Our program would consist of 2 functions,

We will call this main function 10 times in our code

import threading

val = 0 # global variable val

def refresh_val():
    """
    Increment val 100000 times
    """
    global lock, val
    counter = 100000
    while counter > 0:
        val += 1
        counter -= 1


def main():
    global val
    val = 0
    
    # creating threads
    t1 = threading.Thread(target=refresh_val)
    t2 = threading.Thread(target=refresh_val)

    # start threads
    t1.start()
    t2.start()

    # wait until threads complete
    t1.join()
    t2.join()


if __name__ == "__main__":
    for i in range(1,11):
        main()
        print("Step {0}: val = {1}".format(i, val))

Step 1: val = 200000
Step 2: val = 191360
Step 3: val = 200000
Step 4: val = 200000
Step 5: val = 200000
Step 6: val = 199331
Step 7: val = 200000
Step 8: val = 200000
Step 9: val = 157380
Step 10: val = 200000

Let’s perform the above same operation, with the locking mechanism present in threading.

Here is where the threading module introduces 2 methods,

When a lock is acquired by a thread for a shared object, no other thread can make changes to this object at the same time. After a lock is acquired, if another thread attempts to access an object, it would have to wait until the lock is released.

Methods to create a lock

Method 1 :

import threading

lock = threading.Lock() # create a lock
try:
    lock.acquire() # Block the lock
    # code goes here
finally:
    lock.release() # Release the lock

Method 2 :

We can also create a python thread lock using the context manager.

It is recommended to use this method in order to ensure both acquire and release methods are applied whenever a lock is created in python.

import threading

lock = threading.Lock() # create a lock
with lock:
    # code goes here

Example 2 : Threading with Lock in Python

Let’s perform the above same operation, using lock in python to achieve synchronization between threads.

As you can see below, the global variable is being incremented concurrently as expected without any inconsistency.

import threading

val = 0 # global variable val

lock = threading.Lock() # create a lock

def refresh_val():
    """
    Increment val 10000 times
    """
    global lock, val
    counter = 100000
    while counter > 0:
        lock.acquire() # Block the lock
        val += 1
        lock.release() # Release the lock
        counter -= 1


def main():
    global val
    val = 0
    
    # creating threads
    t1 = threading.Thread(target=refresh_val)
    t2 = threading.Thread(target=refresh_val)

    # start threads
    t1.start()
    t2.start()

    # wait until threads complete
    t1.join()
    t2.join()


if __name__ == "__main__":
    for i in range(1,11):
        main()
        print("Step {0}: val = {1}".format(i, val))

Step 1: val = 200000
Step 2: val = 200000
Step 3: val = 200000
Step 4: val = 200000
Step 5: val = 200000
Step 6: val = 200000
Step 7: val = 200000
Step 8: val = 200000
Step 9: val = 200000
Step 10: val = 200000

When to not use Multithreading in Python?

Python Multithreading is not recommended for below mentioned scenarios,

• Not suitable for CPU intensive tasks. Example stated below.
• Having multiple heavyweight threads can slow down your main process.
• Individual threads are not killable.
• Creating too many threads for a single application might make your code longer and process slower.

The following example clearly indicates why threading in python isn’t suitable for even simple CPU bound tasks. Below is a number crunching task which is run using 2 threads and compared with a sequential execution of the same.

import time
from concurrent.futures import ThreadPoolExecutor


def decrement():
    counter = 10000000
    while counter > 0:
        counter -= 1

def execute_parallel():
    start_time = time.time()
    
    with ThreadPoolExecutor(max_workers=2) as executors:
        for _ in range(2):
            executors.submit(decrement) 
            
    print(f"Threaded Execution - Time Taken: {time.time() - start_time}")
    
def execute_serially():
    start_time = time.time()
    
    for _ in range(2):
        decrement()
        
    print(f"Serial Execution - Time Taken: {time.time() - start_time}")

if __name__ == "__main__":
    execute_parallel()
    execute_serially()

Threaded Execution - Time Taken: 1.3582673072814941
Serial Execution - Time Taken: 1.2527658939361572

Conclusion

We can summarise by our learning that Multithreading in Python can be used in cases where you would like to perform multiple I/O bound tasks within the same application accessing some shared objects.

Multithreading in python is more recommended for I/O bound tasks because of the implementation of the Global Interpreter Lock(GIL).

We have also learnt how we can start a pool of threads using ThreadPoolExecutor or ThreadPool class.

To get rid of inconsistency during race conditions, the threading lock can be used.

By now, you should be able to leverage multithreading in python based on your requirements.

Comments and feedback are welcome. Cheers!