Carol Willing Thriving with Python Navigate the pitfalls in a
polyglot world April 21, 2024 @[emailprotected] https://speakerdeck.com/willingc 1
None
polyglot knowing, using, or written in multiple languages
Hi!
Carol Willing Python Core Developer Python Steering Council (three terms)
Python Software Foundation Fellow Jupyter Core Developer 2017 ACM Software System Award Papermill Maintainer pyOpenSci Advisory Board Chan Zuckerberg Open Science Board Quansight Labs Board VP of Engineering, Noteable
How do we succeed in a polyglot world?
Python and ??? Which languages to choose? • Rust, Go,
C, C++ • JavaScript, TypeScript • Java, C# • Haskell, Clojure, Scala • Lua, Perl • Swift and others
Watch your step How do we avoid the pitfalls? Swooning
at shiny Chasing speed Hype and FOMO
None
What should I do?
https://thenewstack.io/a-conversation-with-the-creators-behind-python-java-typescript-and-perl/ Guido van Rossum (Python) James Gosling (Java) Larry Wall
(Perl) Anders Hejlsberg (Pascal, C#, TypeScript) me (Jupyter)
None
1970s BASIC • Bell Labs mainframe BASIC • TRS-80 BASIC
• Apple BASIC Don Slepian, CC BY-SA 4.0 <https://creativecommons.org/licenses/by-sa/ 4.0>, via Wikimedia Commons Early personal computers By FozzTexx - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/ index.php?curid=79580939
1980s Fortran, Pascal, C, LISP, SQL • Mobile phones emerging
• No smartphones • No web, no google • No Linux • No Python https://archive.org/details/byte-magazine-1985-04/ Yes - Arti fi cial Intelligence 😱
My Journey
Paradigm Shi ft s Transistors and mainframes Personal Computers Worldwide
Web, Smartphones, Cloud computing and data science Generative AI and LLMs Cellular service Time
How?
Follow your North Star
Embrace a new musician's mistakes mindset Loud and proud
Take a step, then SCAMPER https://www.designorate.com/a-guide-to-the-scamper-technique-for-creative-thinking/
Understand your
Let's go tackle some pitfalls
Pitfall #1 Chasing speed
Questions to ponder Speed • What to measure? • How
to measure? • What are my goals? • Is it fast enough? • What are the tradeo ff s? Performance Reliability Flexibility Security
3.11, 3.12, 3.13 Use a recent Python • Try the
latest version • Alternatively, try pypy • 3.13 • Experimental freethreading without the gil (global interpreter lock) • JIT compiler • Release date: Tuesday, 2024-10-01 https://docs.python.org/3/whatsnew/index.html
Measure, then optimize Pro fi le • timeit or magic
%timeit • cPro fi le (deterministic pro fi ler) • Pyinstrument (statistical pro fi ler) • py-spy (sampling pro fi ler) • pyperf (https://github.com/psf/pyperf) py-spy basics https://docs.ray.io/en/latest/cluster/kubernetes/k8s-ecosystem/pyspy.html py-spy Search fit (implicit/als.py:159) calculate_similar_artists (lastfm.py:79) fit (implicit/als.py:163) _get_similarity_score (implic.. _get_similarity_score (implicit/recommender_.. argpartition (numpy/core/fromnumeric.py:757) _wrapfunc (numpy/core/fromnumeric.py:51) calculate_similar_artists (lastfm.py:95) similar_items (implicit/recommender_base.py:203) <module> (lastfm.py:161) fl amegraphs to visualize
Data structures • Concurrency / Asynchronous • Vectorization and numpy • Accelerate using best practices in Python • Substitute C, Rust, Go, C++ https://tinkering.xyz/fmo-optimization-story/
Itamar Turner-Trauring and pythonspeed.com Best practices Vectorization: How slow Python
runs fast code | PyData Global 2022 Vectorization: How slow Python runs fast code | PyData Global 2022 Timestamps for video https://pythonspeed.com/datascience/ Faster Data Science: Speed up your data science and scienti fi c computing code
GPUs and JITs • CUDA • numba • cupy Use
to test with and without the decorator: %timeit monte_carlo_pi(1000)
Complements Python and Rust Prototype in Python and move performance
bottlenecks to Rust. - Moshe Zadka • Use PyO3 to wrap Rust code for Python • https://pyo3.rs/v0.21.2/getting-started • maturin develop https://opensource.com/article/23/3/python-loves-rust
Development Speed • Ru ff • Playwright https://bloomberg.github.io/memray/index.html https://bloomberg.github.io/pystack/ Bloomberg
1/2 million fi les, 100 million lines of code, funding PSF https://www.bloomberg.com/company/values/tech-at-bloomberg/python/
Up and running Prototyping Speed • Iterative • 5 lines
of code or less can do meaningful work • Interactive with Jupyter notebooks https://web.mit.edu/music21/
Pitfall #2 Swooning at shiny
None
Grass is greener in another language
https://cheezburger.com/9380419584/1-ring-to-rule-them-all pip poetry pdm conda hatch fl it mamba uv
pixi • Rust - cargo, fast resolver, one stop distribution • JS - lock fi les, tools update everything packaging Maybe
Python: a decade of improvements Maybe not • Pure Python
packages • Thank you PyPA volunteers • Reach out and thank a maintainer • Watch Bernat Gabor's PyTexas talk on helping maintainers • Outreach e ff orts
Science, Python, Community pyOpenSci • Python Package Guide • Tutorials
• Peer reviewed scienti fi c packages • Inclusive open science
Next gen conda pixi
Rust, TypeScript, Python play.ru ff .rs https://github.com/astral-sh/ruff/tree/main/playground Playground for formatting,
AST, tokens, formatter IR
play.ru ff .rs https://github.com/astral-sh/ruff/tree/main/playground AST Abstract Syntax Tree
play.ru ff .rs https://github.com/astral-sh/ruff/tree/main/playground AST Abstract Syntax Tree
Rust and Python in the real world Py03
Python, JavaScript, p5.js JupyterLite, pyodide, WebAssembly https://www.willingconsulting.com/pysplashlive.html JupyterLite runs entirely
in the browser.
None
Testing Playwright for Python • Performant and built using Rust
• Web focus • JS, Python • Examples, docs, and YouTube updates https://playwright.dev/python/
import re from playwright.sync_api import Page, expect def test_has_title(page: Page):
page.goto("https://playwright.dev/") # Expect a title "to contain" a substring. expect(page).to_have_title(re.compile("Playwright")) def test_get_started_link(page: Page): page.goto("https://playwright.dev/") # Click the get started link. page.get_by_role("link", name="Get started").click() # Expects page to have a heading with the name of Installation. expect(page.get_by_role("heading", name="Installation")).to_be_visible()
PWDEBUG=1 pytest -s test_example.py Launch Playwright debugger
Load page and test for link
Execute another test and step
Pitfall #3 Hype and FOMO
import antigravity Use Python. It's remarkable, and you can skip
the TSA line. Wow!
None
None
None
Interview with Scott Hanselmann Dr. Brandeis Marshall We all are
data people.
Tech Trends Source: Forrester Critical Global Issues Source: UN
Timeline to a new technology What if I try... now
basic grounding understand from the beginning 5 years solve a problem best practices automation 6 months more examples peers and their experiences 1 year better tooling better docs emerging packages
None
Re fl ections from our polyglot journey
Awareness Pitfalls Swooning at shiny Chasing speed Hype and FOMO
Keys to success
Expect to make mistakes 1
Take a step, then SCAMPER 2
Understand your "why" 3
Treasure your time and energy 4
Fun builds resilience 5
What's next?
None
Python and the polyglots
Introducing their hit song Thriving at Python and the polyglots
Python and the polyglots Thank you
Python and the polyglots Thank you
Carol Willing Thriving with Python Navigate the pitfalls in a
polyglot world April 21, 2024 The End
