Python

6343 readers

10 users here now

Welcome to the Python community on the programming.dev Lemmy instance!

📅 Events

Past

November 2023

PyCon Ireland 2023, 11-12th
PyData Tel Aviv 2023 14th

October 2023

PyConES Canarias 2023, 6-8th
DjangoCon US 2023, 16-20th (!django 💬)

July 2023

PyDelhi Meetup, 2nd
PyCon Israel, 4-5th
DFW Pythoneers, 6th
Django Girls Abraka, 6-7th
SciPy 2023 10-16th, Austin
IndyPy, 11th
Leipzig Python User Group, 11th
Austin Python, 12th
EuroPython 2023, 17-23rd
Austin Python: Evening of Coding, 18th
PyHEP.dev 2023 - "Python in HEP" Developer's Workshop, 25th

August 2023

PyLadies Dublin, 15th
EuroSciPy 2023, 14-18th

September 2023

PyData Amsterdam, 14-16th
PyCon UK, 22nd - 25th

🐍 Python project:

💓 Python Community:

#python IRC for general questions
#python-dev IRC for CPython developers
PySlackers Slack channel
Python Discord server
Python Weekly newsletters
Mailing lists
Forum

✨ Python Ecosystem:

🌌 Fediverse

Communities

#python on Mastodon
c/django on programming.dev
c/pythorhead on lemmy.dbzer0.com

Projects

Pythörhead: a Python library for interacting with Lemmy
Plemmy: a Python package for accessing the Lemmy API
pylemmy pylemmy enables simple access to Lemmy's API with Python
mastodon.py, a Python wrapper for the Mastodon API

Feeds

founded 1 year ago

MODERATORS

[email protected]

Multiple process or threads? And, why? (programming.dev)

submitted 3 weeks ago by [email protected] to c/[email protected]

16 comments fedilink hide all child comments

(For context, I'm basically referring to Python 3.12 "multiprocessing.Pool Vs. concurrent.futures.ThreadPoolExecutor"...)

Today I read that multiple cores (parallelism) help in CPU bound operations. Meanwhile, multiple threads (concurrency) is due when the tasks are I/O bound.

Is this correct? Anyone cares to elaborate for me?

At least from a theorethical standpoint. Of course, many real work has a mix of both, and I'd better start with profiling where the bottlenecks really are.

If serves of anything having a concrete "algorithm". Let's say, I have a function that applies a map-reduce strategy reading data chunks from a file on disk, and I'm computing some averages from these data, and saving to a new file.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 2 points 3 weeks ago (1 children)

I can’t remember if threads are core bound or not.

On Linux, by default they're not. getcpu(2) says:

   The getcpu() system call identifies the processor and node on which the
   calling thread or process is currently running and writes them into the
   integers pointed to by the cpu and node arguments.  ...

   The  information  placed in cpu is guaranteed to be current only at the
   time of the  call:  unless  the  CPU  affinity  has  been  fixed  using
   sched_setaffinity(2),  the  kernel  might  change  the CPU at any time.
   (Normally this does not happen because the scheduler tries to  minimize
   movements  between  CPUs  to keep caches hot, but it is possible.)  The
   caller must allow for the possibility that the information returned  in
   cpu and node is no longer current by the time the call returns.

[–] [email protected] 1 points 3 weeks ago

Thank you. That's good to know. In my OS architecture lectures, we were introduced to an OS with core bound threads. I can't remember if it was a learning OS or something that really existed, hence my doubts.

Anti Commercial-AI license