Python is the only programming language that has forced me to question what the difference is between an egg and a wheel.
this post was submitted on 06 Nov 2024
192 points (98.0% liked)
Programming
17352 readers
613 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities [email protected]
founded 1 year ago
MODERATORS
No, it's not just you, Python's tooling is a mess. It's not necessarily anyone's fault, but there are a ton of options and a lot of very similarly named things that accomplish different (but sometimes similar) tasks. (pyenv, venv, and virtualenv come to mind.) As someone who considers themselves between beginner and intermediate proficiency in Python, this is my biggest hurdle right now.
Python’s tooling is a mess.
Not only that. It's a historic mess. Over the years, growing a better and better toolset left a lot of projects in a very messy state. So many answers on Stack Overflow that mention easy_install - I still don't know what it is, but I guess it was some kind of proto uv.
Every time I'm doing anything with Python I ask myself if Java's tooling is this complicated or I'm just used to it by now. I think a big part of the weirdness is that a lot of Python tooling is tied to the Python installation whereas in Java things like Maven and Gradle are separate. In addition, I think dependencies you install get tied to that Python installation, while in Java they just are in a cache for Maven/Gradle. And in the horrible scenario where you need to use different versions of Maven/Gradle (one place I was at specifically needed Maven 3.0.3 for one project and a different for a different, don't ask, it's dumb and their own fault for setting it up that way) at least they still have one common cache for everything.
I guess it also helps that with Java you (often) don't need platform specific jar files. But Python is often used as an easy and dynamic scripting interface over more performant, native code. So you don't really run into things like "this artifact doesn't have a 64 bit arm version for python 2" often with Java. But that's not a fault of Python's tooling, it's just the reality of how it's used.
Python developer here. Venv is good, venv is life. Every single project I create starts with
python3 -m venv venv
source venv/bin/activate
pip3 install {everything I need}
pip3 freeze > requirements.txt
Now write code!
Don't forget to update your requirements.txt using pip3 freeze again anytime you add a new library with pip.
If you installed a lot of packages before starting to develop with virtual environments, some libraries will be in your OS python install and won't be reflected in pip freeze and won't get into your venv. This is the root of all evil. First of all, don't do that. Second, you can force libraries to install into your venv despite them also being in your system by installing like so:
pip3 install --ignore-installed mypackage
If you don't change between Linux and windows most libraries will just work between systems, but if you have problems on another system, just recreate the whole venv structure
rm -rf venv (...make a new venv, activate it) pip3 install -r requirements.txt
Once you get the hang of this you can make Python behave without a lot of hassle.
This is a case where a strength can also be a weakness.
pip3 freeze > requirements.txt
I hate this. Because now I have a list of your dependencies, but also the dependencies of the dependencies, and I now have regular dependencies and dev-dependencies mixed up. If I'm new to Python I would have NO idea which libraries would be the important ones because it's a jumbled mess.
I've come to love uv (coming from poetry, coming from pip with a requirements/base.txt and requirements/dev.txt - gotta keep regular dependencies and dev-dependencies separate).
uv sync
uv run <application>
That's it. I don't even need to install a compatible Python version, as uv takes care of that for me. It'll automatically create a local .venv/, and it's blazingly fast.
I've never really spent much time with uv, I'll give it a try. It seems like it takes a few steps out of the process and some guesswork too.
OP sounds like a victim of Python 3, finding various Python 2 projects on the internet, a venv isn't going to help
Okay, now give me those steps but what to do if I clone an already existing repo please
The git repo should ignore the venv folder, so when you clone you then create a new one and activate it with those steps.
Then when you are installing requirements with pip, the repo you cloned will likely have a requirements.txt file in it, so you 'pip install -r requirements.txt'
This is the way
This is the way.
Yes it's terrible. The only hope on the horizon is uv. It's significantly better than all the other tooling (Poetry, pip, pipenv, etc.) so I think it has a good chance of reducing the options to just Pip or uv at least.
But I fully expect the Python Devs to ignore it, and maybe even make life deliberately difficult for it like they did for static analysers. They have some strange priorities sometimes.
uv is good but it needs a little more time in the oven.
For the moment I would definitely recommend poetry if you are not a library developer. Poetry's biggest sin is it's atrocious performance but it has most of the features you need to work with Python apps today.
Why do you say it needs more time in the oven? I've had zero issues with it as a drop-in replacement for Pip in a large commercial project, which is an extremely impressive achievement. (And it was 10x faster.)
I tried Poetry once and it failed to resolve dependencies on the first thing I tried it on. If anything Poetry needs more time in the oven. It also wasn't 10x faster.
I like the idea of uv, but I hate the name. Libuv is already a very popular C library, and used in everything from NodeJS to Julia to Python (through the popular uvloop module). Every time I see someone mention uv I get confused and think they're talking about uvloop until I remember the Astral project, and then reconfirm to myself how much I disapprove of their name choice.
I don't think libuv is really that popular, nor is it that confusing.
But I do agree it's not a very good name. "Rye" is a much better name. Probably too late anyway.
UV is a game changer for python.
I hated the tooling until I found it.
load more comments
(4 replies)
The reason you do stuff in a venv is to isolate that environment from other python projects on your system, so one Python project doesn’t break another. I use Docker for similar reasons for a lot of non-Python projects.
A lot of Python projects involve specific versions of libraries, because things break. I’ve had similar issues with non-Python projects. I’m not sure I’d say Python is particularly worse about it.
There are tools in place that can make the sharing of Python projects incredibly easy and portable and consistent, but I only ever see the best maintained projects using them unfortunately.
Python's packaging is not great. Pip and venvs help but, it's lightyears behind anything you're used to. My go-to is using a venv for everything.
It's something of a "14 competing standards" situation, but uv seems to be the nerd favourite these days.
I still do the python3 -m venv venv && source venv/bin/activate
How can uv help me be a better person?
- let
pyproject.tomltrack the dependencies and dev-dependencies you actually care about
- dependencies are what you need to run your application
- dev-dependencies are not necessary to run your app, but to develop it (formatting, linting, utilities, etc)
- it can track exactly what's needed ot run the application via the
uv.lockfile that contains each and every lib that's needed. - uv will install the needed Python version for you, completely separate from what your system is running.
uv syncanduv run <application>is pretty much all you need to get going- it's blazingly fast in everything
Thank you for explaining so clearly. Point 3 is indeed something I've ran into before!
load more comments
(5 replies)
load more comments
(6 replies)
I've started using poetry and the experience has improved.
Python is hacky, because it hacks. There’s a bunch of ways you can do anything. You can run it on numerous platforms, or even on web assembly. It’s not maintained centrally. Each “app” you find is just somebodies hack project they’re sharing with you for fun.
load more comments
(6 replies)
You re not stupid, python's packaging & versionning is PITA. as long as you write it for yourself, you re good. As soon as you want to share it, you have a problem
load more comments
(1 replies)
everyone focuses on the tooling, not many are focusing on the reason: python is extremely dynamic. like, magic dynamic you can modify a module halfway through an import, you can replace class attributes and automatically propagate to instances, you can decompile the bytecode while it's running.
combine this with the fact that it's installed by default and used basically everywhere and you get an environment that needs to be carefully managed for the sake of the system.
js has this packaging system down pat, but it has the advantage that it got mainstream in a sandboxed isolated environment before it started leaking out into the system. python was in there from the beginning, and every change breaks someone's workflow.
the closest language to look at for packaging is probably lua, which has similar issues. however since lua is usually not a standalone application platform it's not a big deal there.
view more: next ›