Gunicorn with Uvicorn inside

I'm up to reading the Gunicorn and Uvicorn source code and playing with the parameters to see the actual process/thread/coroutine management of those. I will write a detailed blog post about it soon but so far I wanna explain to you why it's recommended to use the Gunicorn web server with Uvicorn worker class.

The Gunicorn provides powerful worker management with lots of customizations. Gunicorn manages workers, and workers manage Python web applications. There are two types of workers which are sync and async. The term "async" here is a bit different than the async functions in Python. It's about the way worker serves the requests.

There are a couple of default worker classes in Gunicorn such as sync, eventlet, gevent, torando and gthread. There are differences between request and web application handling in these workers that I will explore in the upcoming blog post. However, none of them can run the async functions in Python. So you will lose the power of this handy process manager for just being in async code style.
Here is the moment that Uvicorn comes. Although the Uvicorn is a standalone web server that runs ASGI applications in Python, but it does not have the abilities of Gunicorn. The great thing that developers of Uvicorn have done is that they developed a Uvicorn worker class based on the basic Gunicorn worker class. So you can easily run the Gunicorn to manage Uvicorn workers, and use this worker to serve incoming requests with async web applications.

To do that you can simply run this command:

gunicorn example:app \-w 4 \-k uvicorn.workers.UvicornWorker

This command will spawn 4 Uvicorn workers and loads the ASGI application into them. So you have the performance of Gunicorn and Uvicorn together.

You might ask if we should avoid running our apps only with Uvicorn? The answer is not the same for different situations. Sometimes you wanna keep the running process of a web application flat and simple to put it as a single processing unit into Kubernetes pods or Docker containers for better management and debugging. So you should see your structure first.

@OhMyTask

An optimal system is made of an aware manager and good workers.

See how distributed you areWhen it comes to writing apps that use concurrency or parallelism, this question is usually asked: Am I really doing concurrent/parallel?

There are some profiling tools that will help you make a good report of the execution of your app. In Python, personally I use viztracer which is very handy. You just need to run your program like:

viztracer myapp.py

And it will profile the execution of your program and trace each of process, thread and coroutines. And it stores the result in a json file.
Then you can see the result in a nice web interface using this command:

vizviewer result.json

It happened many times that I assumed my tiny piece of code will perfectly doing well at concurrency, parallelism but I checked the running process and found out I was wrong!

Don’t be a code-delivery programmer. Double check your insights.

@OhMyTask

ASGI Lifespan

First of all, let’s talk a bit about the WSGI, the core of each modern Pythonic web framework.
Despite the importance of this protocol, it’s really simple. All you need is a callable (function or class) that follows a specific signature. When you run the application using a WSGI-friendly web server like the Gunicorn, for each HTTP request the web server calls your defined callable with the data of that incoming request. And your callable should respond to that in a specific format.
For example:

def simple\_app(environ, start\_response): status = '200 OK' response\_headers = [('Content\-type','text/plain')] start\_response(status, response\_headers) return ['Hello world!\n']

The simple_app function receives environ that contains the data of incoming HTTP requests. The start_response is a function that is used for announcing web server that we’re sending the response data or even revoking the connection.

ASGI (Asynchronous Server Gateway Interface) is an async version of that. But it can do more. Thanks to the async-await of Python. It can keep the connection open for a long time for protocols like WebSocket. It can receive and send data a couple of times.
Here’s an ASGI app that servers WebSocket:

async def application(scope, receive, send): event = await receive() ... await send({"type": "websocket.send", ...})

We can call multiple functionalities on web servers using the type attribute. In this example we used websocket.send to send data through the WebSocket protocol.

Now that we understand the structure of ASGI applications. Assume we're gonna develop a simple web app with an AI model. We need to load the model in memory before serving any request. So, how exactly did we find out that our application started serving requests? How to do operations before that? When the lifetime of our ASGI application begins and when it finishs?

Here the lifespan comes into the game. When an ASGI-friendly web server like Uvicorn loads the application and wants to start the server, It calls our application, with the scope of lifespan. And in the application we notice that it's the beginning of our life. So we can prepare anything we need for the lifetime of the application, such as loading AI models, making db connections, etc.

async def app(scope, receive, send): if scope['type'] == 'lifespan': while True: message = await receive() if message['type'] == 'lifespan.startup': ... \# Do some startup here! await send({'type': 'lifespan.startup.complete'}) elif message['type'] == 'lifespan.shutdown': ... \# Do some shutdown here! await send({'type': 'lifespan.shutdown.complete'}) return else: pass \# Handle other types

The above example shows you a sample of managing application lifetime using lifespan scope.

This feature has been added to the ASGI since 2018 but it wasn't supported in ASGI-based web frameworks for some reasons. But now, they're replacing their old-fashioned lifetime management methods with this fancy lifespan.
For example, you can now pass an async context manager to a FastAPI app as lifespan. And it automatically handles the communication with web server.

```
@asynccontextmanager
async def lifespan(app: FastAPI):
create_db_connection()
yield
close_db_connection()

app = FastAPI(lifespan=lifespan)
```

In the above example we easily managed the operations before and after application lifetime. Learn more about lifespan in FastAPI here.

This is a handy feature. This unifies all AGSI web servers to act similarly in managing lifetime. And you don't have to write lifetime manager based on the web server you use. These are the challenges with WSGI applications.

@OhMyTask

The Creation of App
@OhMyTask

Suppose you're looking for a "complete enough" course for async-await programming on Python. Who's better than an active contributor of Python to teach you?

Take a look at this amazing ~6 hours course made by Łukasz Langa (Python core developer)

https://youtube.com/playlist?list=PLhNSoGM2ik6SIkVGXWBwerucXjgP1rHmB&si=eyhr590l6I9DlIZi

@OhMyTask

Decentralized Brain of Nature

What is your first reaction when you see an ant or a group of them? Scream? Get mad? Grap something and try to kill them?

OK. Chill out for one minute. Let's observe. Imagine you see a group of ants (more often) and they're living their life, stealing your tiny pieces of food and going back to their colony.
Have you ever wondered how they're moving in a specific pattern without chaos? Are they talking to each other and yelling: Hey ant-186, look where ant-240 is going and follow him? Are they sending wireless messages to each other?

Nature knows its way. The group of ants has a decentralized brain or more scientific Swarm intelligence. When they are moving, they emit chemicals called Pheromone. These pheromones help ants find the path to a food source and also return to the nest in an efficient manner.
The pheromone has a lifetime and the more time passes, the smell of it gets weaker. So the ants know this pheromone is new or old. The more amount of pheromone on a route, the stronger smell, and probably the more efficient and acceptable path to use.

What's the point here? Ok. Let's see what challenges we have in our systems. Imagine we are in a huge and complex internet network with lots of nodes, lots of connections, lots of routers, etc.
We have groups of packets from node A to node B. There are a couple of ways to reach the node B. In a chaotic strongly connected network. How do you efficiently find the best path to Node B?

Or imagine there has been a fire accident and you're sending a couple of rescue drones to find injured people. The fire is getting worse. Drones are inside the building. How do you manage these drones to don't get damaged and most importantly, find the people inside the building as fast as they can?

Two above challenges and many challenges like those couldn't be solved by a centralized management system. They could but it's not that efficient and reliable. So here the algorithms of nature would help us. Swarm intelligence algorithms such as ACO (Ant Colony Optimization) we learned earlier could save lots of lives.

The deeper you look at the phenomena, the more you avoid erasing or killing them. Just you learn from the behavior.

@OhMyTask