title: Why hasn't Python's Type Hint become popular?
type: tags
date: 2020-03-20 16:00:00
tags: [Python, programming, random thoughts]
categories: [programming, Python, random thoughts]
toc: true#
I saw an interesting question on Zhihu, Why is TypeScript so popular, but it's rare to see people writing Python with type annotations?
Although I couldn't resist answering on Zhihu, just in case, I decided to expand and update it on my blog.
BTW, I've been really exhausted recently, so I'm writing an article to relax.
Introduction#
The answer to this question is actually quite simple: historical baggage and ROI. Before understanding why this phenomenon exists, we first need to understand what Type Hint can bring us, and then we need to understand the history of Type Hint.
As of now (March 2020), Type Hint can bring us visible benefits:
- Through annotations, combined with IDE support, it can improve the coding experience.
- With the support of tools like mypy/pytype, we can integrate static type checking into the CI/CD process.
- With the support of pydantic and many modern frameworks, we can reduce a lot of repetitive work.
You might think that Python Type Hint has been mature since the introduction of PEP 484 in Python 3.5. But in reality, it has been a much shorter time than you think.
Well, let's review the key milestones in the development history of Type Hint.
- PEP 3107 Function Annotations
- PEP 484 Type Hints
- PEP 526 Syntax for Variable Annotations
- PEP 563 Postponed Evaluation of Annotations
PEP 3107#
As mentioned earlier, the time when everyone first became aware of Type Hint should be when it was proposed in September 2014 and accepted in May 2015 in PEP 484. But in fact, the prototype was much earlier, and the syntax of PEP 484 actually came from PEP 3107, which was proposed in 2006 and introduced in Python 3.0. Please refer to PEP 3107 -- Function Annotations for more details.
In PEP 3107, there is a description of the goals of this proposal:
Because Python's 2.x series lacks a standard way of annotating a function's parameters and return values, a variety of tools and libraries have appeared to fill this gap. Some utilise the decorators introduced in "PEP 318", while others parse a function's docstring, looking for annotations there.
This PEP aims to provide a single, standard way of specifying this information, reducing the confusion caused by the wide variation in mechanism and syntax that has existed until this point.
In simple terms, in order to add additional metadata to a function's parameters or return values, people used various methods, such as using decorators introduced in "PEP 318" or parsing a function's docstring. In order to alleviate this confusion, the community decided to introduce a new syntax to allow users to easily add additional information to parameter signatures and return values.
The final syntax is as follows:
def foo(a: 'x', b: 5 + 6, c: list) -> max(2, 9):
pass
Does it look familiar? Yes, PEP 3107 actually laid the foundation for subsequent Type Hint.
- It can be annotated.
- It is part of the function/method information and can be inspected.
- It is evaluated at runtime.
But a new question arises: why is this proposal often overlooked? Or, we need to look at it from a specific point in time.
This proposal can be traced back to 2006, and it was confirmed in PEP 3000, which is probably the most famous proposal in Python history (declaring the birth of Python 3). It was officially released in 2008.
At that time, PEP 3107 faced two problems:
- From 2006 to 2008, the community's main focus was on discussing why we needed Python 3 and why we should migrate to Python 3.
- PEP 3107 only told everyone that they could annotate and easily access the annotation information, but how to abstract the representation of a type, such as a list of integers, still relied on the community's own development. In other words, it was left to the community to figure it out.
Problem 1 had no solution and could only be pushed forward with time. Problem 2 led to the birth of PEP 484.
PEP 484#
PEP 484 is a proposal that most people should have some understanding of, so I won't describe its specific content here.
The significance of PEP 484 is that it builds a reasonable abstraction of Python's type system on top of the syntax and foundation established by PEP 3107. This is also the important product "typing". Only at this point did Python's type hint have a basic official specification and achieve basic usability. This happened in September 2015 (September 13, Python 3.5.0 officially released).
But in reality, PEP 484 at this time can only be said to be basically usable. Let me give you a few examples that have been criticized.
First, let's look at some code:
from typing import Optional
class Node:
left: Optional[Node]
right: Optional[Node]
This code is actually very simple, it describes a standard binary tree node. However, in PEP 484, this code exposes two problems:
- Variables cannot be annotated. As I mentioned earlier, PEP 484 is essentially an extension of PEP 3107, and at this time, the scope of hints is limited to functions/methods. In the above code, in the 3.5 era, I couldn't annotate my left and right variables, which are basic elements of a programming language. So to some extent, we can say that this type hint functionality is not closed.
- Circular references, literally, how to solve the problem of circular references in Type Hint on the community/StackOverflow has been a headache. Community: What the fuck?
Fortunately, the Python community realized this problem and introduced two proposals to solve these problems.
PEP 526#
Problem 1 led to the birth of PEP 526 -- Syntax for Variable Annotations in August 2016, which was accepted in September 2016. It was implemented in BPO-27985 in September 2016. In my impression, this should be one of the PEPs in the Python community with the least controversy, fastest acceptance, and fastest implementation.
In PEP 526, Python officially allows everyone to annotate variables, whether they are class attributes or ordinary variables.
class Node:
left: str
This is allowed.
def abc():
a:int = 1
This is also allowed.
Based on this proposal, the Python official also promoted the implementation of PEP 557 -- Data Classes, but that's another story.
To sum up, PEP 526 only solves problem 1 above and does not solve problem 2. This will be addressed by PEP 563.
PEP 563#
To solve the problem of circular references, Python introduced PEP 563 -- Postponed Evaluation of Annotations in September 2017, which was proposed by the community and accepted in November 2017. It was implemented in GH-4390 in January 2018.
After PEP 563, we can write our code like this:
from typing import Optional
class Node:
left: Optional["Node"]
right: Optional["Node"]
Yes, the two problems in PEP 484 are finally solved.
Conclusion#
PEP 563 can be seen as an important turning point. Python only began to have a complete ecosystem and production usability after January 2018. If we consider the release version, it should be after June 2018, when Python 3.7 was officially released.
After Python 3.6/7, the community began to build an ecosystem around Type Hint. For example, using PEP 526 to efficiently validate data formats, see pydantic.
By the way, this is also a popular new framework (and my favorite framework) called FastAPI.
Major companies have also started to follow suit. For example, Google's pytype and Microsoft's pyright provide support on VSCode.
There are also many excellent libraries like starlette.
Only at this point did the true power of Python + Type Hint begin to emerge. This is when we can start to answer the question, "Why should I switch to Type Hint?" I guess it's not just because it's fun to write in an IDE.
When making technical decisions, we must consider the benefits it can bring us, in other words, the ROI, rather than simply because we like it.
From this perspective, it has only been about a year and a half to two years. For a user habit formation cycle, this is too short. Not to mention that there are still a lot of Python 2 code out there.
By comparison, TypeScript was released in October 2012, version 0.8, and at that time, TypeScript had a relatively complete type system.
TS took 8 years, and Python may still have a long way to go.
Of course, this answer only discusses this question from a technical and historical perspective. Other factors, including the community's bargaining and compromise, are not within the scope of this answer. If you are interested, you can go to python-ideas, python-dev, and discuss-python to find discussions about these proposals in history, which are very interesting.
Finally, one reason for TypeScript's success is that it has a good father and its father has money (escaping).
Well, that's about it. I've been exhausted from work recently, so I can only write some rubbish articles to relieve stress and calm my mood.