Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Getting Started with Statically Typed Programmi...

Getting Started with Statically Typed Programming in Python 3.10 / PyConUS2022

Peacock

May 01, 2022
Tweet

More Decks by Peacock

Other Decks in Programming

Transcript

  1. Getting Started with Statically Typed Programming in Python 3.10 Peacock

    (Yoichi Takai), at PyCon US 2022 (2022/05/01)
  2. Slides: slides.peacock0803sz.com/pyconus2022/index.html Expected: Google Chrome Feel free to Take pictures

    or Screenshots and Tweet Hashtag: #pyconus2022 3 / 57 ` ` ^ Slides ^
  3. Self-Introduction Name: Peacock / Yoichi Takai Social media IDs: peacock0803sz

    Twitter, GitHub, Facebook… Please call me Peacock 4 / 57 ` `
  4. Hobbies & Favourites Playing the Clarinet Listening Music (Most is

    classical) Skiing , Gadgets… 21-years old, This is my first trip abroad 5 / 57
  5. My Works and Communities Company: CMScom (Full-time since 2020/06 ~)

    Web application developer, Flask / Pyramid / Plone etc… Experience learning Haskell and TypeScript Now, reading "Types and Programming Languages" As known as "TaPL" Member of PloneJP (Plone User’s Group in Japan) 6 / 57
  6. My Activities Related to PyCon JP Staff of PyCon JP

    (since 2020 - ), 2022: Vice-Chair Operating Member of PyCon JP Association Director of PyCon JP TV It’s the YouTube live about PyCons and local events held once a month 7 / 57
  7. Today’s Topics 1. Why do I talk about typing? 2.

    First step of typing, How to write basically That is what I most want to say 3. Generics, User-Defined types (Best practice included) 4. Updates overview recently & Backward compatibility for 3.9 or before 5. Overview of new features on 3.10 8 / 57
  8. Why do I talk about Typing? It’s been 7 years

    since typing appeared In Python 3.5, at 2015 Many big PEPs were adopted and updated over the years Even now, I think many people don’t know where to start Because there is little coherent information 9 / 57
  9. What I will NOT talk about Developing library with typing

    Configures and options of mypy How to use them in CI, ex: GitHub actions, Circle CI and etc… History of type hinting Implementation of typing, mypy Abstract Syntax Tree (AST) 10 / 57
  10. What makes you happy? It knows the type when you

    reference it in the editor It gets angry when I try to give it the wrong one The completion will work when accessing the return value of a function using a dot. 12 / 57
  11. With the Typing and, the editor can tell the argument

    is wrong Editor tells a wrong argument 14 / 57
  12. In a code review, without Typing… Boss < What type

    does this function return? You < Humm… str or False or None …? Boss < THAT’S TOO MANY TYPES! You < :-( 4 else: return post_id # this is str 15 / 57 1 def need_new_post(): 2 if ...: retrun None 3 elif ...: retrun False
  13. With Typing… Boss < It looks like this function may

    return 3 types… Isn’t that too much? You < I see. That could be a bad design. Let me fix it. Boss < Sure, please. 1 def need_new_post() -> None | False | str: 16 / 57 2 if ...: retrun None 3 elif ...: retrun False 4 else: return post_id # this is str
  14. Let’s start with function definitions Nothing too difficult! Just write

    type After the arguments, write colon and the type Before the colon at the end of the function definition, write an arrow and the type 17 / 57
  15. Using built-in types bool , bytes , float , int

    , str You don’t need to do anything to use them. None : Used for functions that return nothing. 18 / 57 ` ` ` ` ` ` ` ` ` ` ` `
  16. Escaping from complex type puzzles Any Can hold instances of

    any type. It’s better not to use it Import and use from typing when necessary. 1 from typing import Any 2 very_dangerous_last_resort: Any 19 / 57 ` ` ` `
  17. (Since 3.9) Generics in standard Collections Before 3.9, from typing

    import Dict, List … Now, it’s depreciated Use dict , frozenset , list , set , tuple Collections can be written with [] for the type inside. ref: Official Documentation 20 / 57 ` ` ` ` ` ` ` ` ` ` ` ` ` `
  18. Module Availabilities in standard library… collections : (ex: deque, defaultdict)

    collections.abc : Iterable , Callable , or other Protocol-Related re : Regular Expressions contextlib : Context-Related items 21 / 57 ` ` ` ` ` ` ` ` ` ` ` `
  19. Using types from collections.abc differently There are many types in

    collections.abc . It’s better to use a collection with a few methods to increase portability. Ex. Iterable is better than list . It’s a good idea to look at the methods used in your functions. The figure on the next page shows the relationship… 22 / 57 ` ` ` ` ` ` ` `
  20. The differences between tuple and other Sequences tuple is fixed

    up to the length of information Specify the type for the number of elements Or you can mix types, such as tuple[int, str, float] . 24 / 57 ` ` ` `
  21. A sequence, such as a list , has the same

    constraint for all elements Can be used regardless of the length of the Sequence by setting only one. 25 / 57 ` `
  22. Union (multiple types) union can be represented by | since

    3.10 1 def square(number: int | float) -> int | float: 7 # Compatible with typing.Union 8 int | str == typing.Union[int, str] 27 / 57 ` ` ` ` 2 return number ** 2 1 # Nested unions are flattened 2 (int | str) | float == int | str | float 3 # Redundant types are removed 4 int | str | int == int | str 5 # Orders are ignored 6 int | str == str | int
  23. Optional type Shorthand, Optional[T] is equivalent to Union with None.

    Behaves just like Union: T | None If you use it in a function return value or something, it will propagate, so be careful when you use it. 3 age: Optional[int] 4 age = 17 5 age = None # This is also valid 28 / 57 ` ` ` ` 1 from typing import Optional 2
  24. Avoid using Optional as much as possible 6 return None

    When you use the function, you might write another guard and return None . 29 / 57 1 def get_content() -> str | None: 2 r = request.get("https://example.com") 3 if r.status_code != 200: 4 # Guard section (early return) 5 logging.warning("Response: %d", r.status_code) 7 return r.text ` `
  25. In this case… It would be cleaner to raise a

    raise RuntimeError . The cost of raising exceptions in Python is (relatively) low Little performance impact, unlike Java. The lack of null-safe methods in Python is also a factor, Unlike JavaScript. But, if there were such methods, they would be abused. 30 / 57 ` `
  26. Callable (callable objects) 13 return func(*args, **kw) 31 / 57

    1 from collections.abc import Callable # since 3.9 2 from functools import wraps 3 4 def validate(func: Callable 5 ) -> Callable[..., Callable]: 6 @wraps(func) 7 def wrapper(*args, **kw) -> Callable: 8 try: 9 j = request.json 10 if j is None: raise BadRequest 11 except BadRequest: 12 return abort(400) 14 return wrapper
  27. User-defined Generic types Example: a generic mapping type 13 except

    KeyError: return default 32 / 57 1 from typing import TypeVar, Generic 2 KT, VT = TypeVar("KT"), TypeVar("VT") 3 4 class Mapping(Generic[KT, VT]): 5 def __getitem__(self, key: KT) -> VT: pass 6 7 X, Y = TypeVar("X"), TypeVar("Y") 8 9 def lookup_name( # Using Mapping 10 mapping: Mapping[X, Y], key: X, default: Y 11 ) -> Y: 12 try: return mapping[key]
  28. What is the __future__ module? Using typing new feature in

    the older versions, write from __future__ import annotations In addition to typing, it was also used to call 3.x features in 2.x. ex) print_func , unicode_literals etc … ref: __future__, future statement 34 / 57 ` ` ` ` ` ` ` `
  29. Recent Python updates Ver. Status Release EoS Schedule Main new

    feature 3.11 Alpha 7 2022-10 2027-10 PEP 619 Performance tuning 3.10 Bugfix 2021-10 2026-10 PEP 619 Pattern Matching 3.9 Bugfix 2020-10 2025-10 PEP 596 Union Operators 3.8 Security 2019-10 2024-10 PEP 569 = in f-string 3.7 Security 2018-06 2023-06 PEP 537 dataclasses 2.7 EoL 2010-07 2020-01 PEP 373 Improving Decimals https://www.python.org/downloads/ 36 / 57 ` `
  30. New Features on Type Hints in 3.10 PEP 604: New

    Type Union Operator PEP 612: Parameter Specification Variables PEP 613: TypeAlias PEP 647: User-Defined Type Guards See Also: https://docs.python.org/3/whatsnew/3.10.html#new- features-related-to-type-hints 37 / 57
  31. PEP 604: New Type Union Operator | The union above

    type can be used as an operator. You can also use it when asking isinstance() . 1 int | str == typing.Union[int, str] 2 # Compatible with typing.Union Optional[T] can also be written as T | None using this. 38 / 57 ` ` ` ` ` ` ` `
  32. PEP 612: Parameter Specification Variables Motivation Needed a way to

    represent a function that has the same arguments as the specified function 1 from typing import Callable, TypeVar 2 R = TypeVar("R") 3 4 def add_logging(f: Callable[..., R] 5 ) -> Callable[..., R]: 6 def inner(*args: object, **kw: object) -> R: 7 log_to_database() 8 return f(*args, **kwargs) 9 return inner 39 / 57
  33. 1 @add_logging 2 def takes_int_str(x: int, y: str) -> int:

    3 return x + 7 4 5 await takes_int_str(1, "A") 6 await takes_int_str("B", 2) # fails at runtime Approach Adding an argument type called ParameterSpecification solves the problem. It can be used with Callable to behave like a generic callable object. 40 / 57 ` ` ` `
  34. Before 11 12 @add_logging 13 def foo(x: int, y: str)

    -> int: 14 return x + 7 41 / 57 1 from typing import Callable, TypeVar 2 Ps, R = TypeVar("Ps"), TypeVar("R") 3 4 def add_logging(f: Callable[Ps, R] 5 ) -> Callable[Ps, R]: 6 # args: tuple...?, kwargs: dict...? 7 def inner(*args, **kwargs) -> R: 8 log_to_database(*args, **kwargs) 9 return f(*args, **kwargs) 10 return inner
  35. After 10 11 @add_logging 12 def foo(x: int, y: str)

    -> int: return x + 7 42 / 57 1 from typing import Callable, ParameterSpecification, T 2 Ps, R = ParameterSpecification("Ps"), TypeVar("R") 3 4 def add_logging(f: Callable[Ps, R] 5 ) -> Callable[Ps, R]: 6 def inner(*args: Ps.args, **kwargs: Ps.kwargs) -> R: 7 log_to_database(*args, **kwargs) 8 return f(*args, **kwargs) 9 return inner
  36. PEP 613: TypeAlias Motivation We consider global variables without type

    hints to be type aliases. This tends to cause problems with forwarding references, scoping, etc. So, we’re going to make it possible to explicitly define type aliases. 43 / 57
  37. Approach Add a new typing.TypeAlias You can write like this

    T: TypeAlias = int You can write like: T: TypeAlias = "int" Using ForwardReference 44 / 57 ` ` ` ` ` `
  38. Before & After 7 x: TypeAlias = int # type

    alias 8 x: TypeAlias = “MyClass” # type alias 45 / 57 1 x = 1 # untyped global expression 2 x: int = 1 # typed global expression 3 4 x = int # untyped global expression 5 x: Type[int] = int # typed global expression 6
  39. PEP 647: User-Defined Type Guards Motivation Type checker tools use

    a technique called type narrowing to determine the type of information. 5 else: pass # Type of val is narrowed to None 46 / 57 1 def func(val: Optional[str]): 2 # "is None" type guard 3 if val is not None: 4 pass # Type of val is narrowed to str
  40. However, that will not work as intended if the user

    function is used. 6 if is_str_list(val): print(" ".join(val)) TypeGuard allows you to define user-defined type guards via the new typing. 47 / 57 1 def is_str_list(val: List[object]) -> bool: 2 """Determines whether all objects in the list are st 3 return all(isinstance(x, str) for x in val) 4 5 def func1(val: List[object]): 7 # => Error: invalid type ` `
  41. 1 from typing import TypeGuard 2 3 def is_str_list(val: List[object]

    4 ) -> TypeGuard[List[str]]: 5 return all(isinstance(x, str) for x in val) And, type narrowing works like next: 48 / 57
  42. 10 else: reveal_type(val) # OneOrTwoStrs 49 / 57 1 def

    is_two_element_tuple( 2 val: Tuple[str, ...] 3 ) -> TypeGuard[Tuple[str, str]]: 4 return len(val) == 2 5 6 OneOrTwoStrs = Union[Tuple[str], Tuple[str, str]] 7 def func(val: OneOrTwoStrs): 8 if is_two_element_tuple(val): 9 reveal_type(val) # Tuple[str, str]
  43. New Feature on Type Hints in 3.11 Python 3.11 is

    focused on performance tuning, but there are also new features in Typing… PEP 673: Self Type, It’s a way to annotate methods that return an instance of their class 50 / 57
  44. Motivation Calling set_scale on a subclass of Shape , the

    type checker still infers the return type to be Shape . 7 Circle().set_scale(0.5).set_radius(2.7) 8 # => error: Shape has no attribute "set_radius" 51 / 57 ` ` ` ` ` ` 1 class Circle(Shape): 2 def set_radius(self, r: float) -> Circle: 3 self.radius = r 4 return self 5 6 Circle().set_scale(0.5) # *Shape*, not Circle
  45. Workaround, but it’s unintuitive… 15 Circle().set_scale(0.5).set_radius(2.7) # => Circle 52

    / 57 1 from typing import TypeVar 2 TShape = TypeVar("TShape", bound="Shape") 3 4 class Shape: 5 def set_scale(self: TShape, scale: float 6 ) -> TShape: 7 self.scale = scale 8 return self 9 10 class Circle(Shape): 11 def set_radius(self, radius: float) -> Circle: 12 self.radius = radius 13 return self 14
  46. typing.Self resolves that unintuitively 12 Circle().set_scale(0.5) # => It would

    be *Circle*! 53 / 57 ` ` 1 from typing import Self 2 class Shape: 3 def set_scale(self, scale: float) -> Self: 4 self.scale = scale 5 return self 6 7 class Circle(Shape): 8 def set_radius(self, radius: float) -> Self: 9 self.radius = radius 10 return self 11
  47. Summary 1. Benefits, starting with def , Built-in types 2.

    Standard collections (since 3.9) 3. Collections, Union, Optional, Callable, User-defined Generics 4. Updates Overview, __future__ for compatibility 54 / 57 ` ` ` `
  48. 5. New features on Python 3.10 and 3.11 1. 3.10:

    New Type Union Operator, Parameter Specific Variables, TypeAlias, User-Defined Type Guards 2. 3.11: Self Type 55 / 57
  49. Pages I used for reference (Thanks) There are links that

    I referenced. Thank you! https://docs.python.org/3/library/typing.html https://docs.python.org/3.10/whatsnew/3.10.html https://docs.python.org/3.11/whatsnew/3.11.html https://future-architect.github.io/articles/20201223 (ja) https://qiita.com/tk0miya/items/931da13af292060356b9 (ja) https://qiita.com/tk0miya/items/1b093c2eee7668fffb62 (ja) https://qiita.com/tk0miya/items/a27191f5d627601930ed (ja) 56 / 57