Programming

Python is the only programming language that has forced me to question what the difference is between an egg and a wheel.

No, it's not just you, Python's tooling is a mess. It's not necessarily anyone's fault, but there are a ton of options and a lot of very similarly named things that accomplish different (but sometimes similar) tasks. (pyenv, venv, and virtualenv come to mind.) As someone who considers themselves between beginner and intermediate proficiency in Python, this is my biggest hurdle right now.

Python developer here. Venv is good, venv is life. Every single project I create starts with

python3 -m venv venv

source venv/bin/activate

pip3 install {everything I need}

pip3 freeze > requirements.txt

Now write code!

Don't forget to update your requirements.txt using pip3 freeze again anytime you add a new library with pip.

If you installed a lot of packages before starting to develop with virtual environments, some libraries will be in your OS python install and won't be reflected in pip freeze and won't get into your venv. This is the root of all evil. First of all, don't do that. Second, you can force libraries to install into your venv despite them also being in your system by installing like so:

pip3 install --ignore-installed mypackage

If you don't change between Linux and windows most libraries will just work between systems, but if you have problems on another system, just recreate the whole venv structure

rm -rf venv (...make a new venv, activate it) pip3 install -r requirements.txt

Once you get the hang of this you can make Python behave without a lot of hassle.

This is a case where a strength can also be a weakness.

Yes it's terrible. The only hope on the horizon is uv. It's significantly better than all the other tooling (Poetry, pip, pipenv, etc.) so I think it has a good chance of reducing the options to just Pip or uv at least.

But I fully expect the Python Devs to ignore it, and maybe even make life deliberately difficult for it like they did for static analysers. They have some strange priorities sometimes.

The venv stuff is pretty annoying, I agree.

Python's packaging is not great. Pip and venvs help but, it's lightyears behind anything you're used to. My go-to is using a venv for everything.

How to improve Python packaging, or why fourteen tools are at least twelve too many

Still relevant.

The reason you do stuff in a venv is to isolate that environment from other python projects on your system, so one Python project doesn’t break another. I use Docker for similar reasons for a lot of non-Python projects.

A lot of Python projects involve specific versions of libraries, because things break. I’ve had similar issues with non-Python projects. I’m not sure I’d say Python is particularly worse about it.

There are tools in place that can make the sharing of Python projects incredibly easy and portable and consistent, but I only ever see the best maintained projects using them unfortunately.

It's something of a "14 competing standards" situation, but uv seems to be the nerd favourite these days.

I've started using poetry and the experience has improved.

Python is hacky, because it hacks. There’s a bunch of ways you can do anything. You can run it on numerous platforms, or even on web assembly. It’s not maintained centrally. Each “app” you find is just somebodies hack project they’re sharing with you for fun.

You re not stupid, python's packaging & versionning is PITA. as long as you write it for yourself, you re good. As soon as you want to share it, you have a problem

everyone focuses on the tooling, not many are focusing on the reason: python is extremely dynamic. like, magic dynamic you can modify a module halfway through an import, you can replace class attributes and automatically propagate to instances, you can decompile the bytecode while it's running.

combine this with the fact that it's installed by default and used basically everywhere and you get an environment that needs to be carefully managed for the sake of the system.

js has this packaging system down pat, but it has the advantage that it got mainstream in a sandboxed isolated environment before it started leaking out into the system. python was in there from the beginning, and every change breaks someone's workflow.

the closest language to look at for packaging is probably lua, which has similar issues. however since lua is usually not a standalone application platform it's not a big deal there.