Hardware

CPUs and GPUs are both Turing complete, so from a computation perspective, they can technically do the same things. However each is optimized for a different set of metrics, so it may be wise to target a particular workload to one over the other.

The CPU is optimized for minimum latency, aka maximum single threaded performance. This is really important for a lot of workloads.

GPUs are optimized for maximum throughput and are willing to suffer higher latency to achieve it. There are many architectural differences as a result of this goal, for example: GPUs don't have branch prediction, they also don't have out of order processing, and also have much smaller caches per computational unit. All of these saves a ton of transistors, which can be used to add more cores.

I mean if you're creative enough, probably nothing.

This is kinda like asking what can I do on a lathe that I can't do on a mill. It's more what's better suited to be done on one or the other.

CPUs are more generalised; they have a deep and complex instruction set and feature list. While GPUs are shallower and far more specialised, but do tasks that parallellalise more readily... Like calculating a metric shitload of triangles.

You can see CPUs used to 'push pixels' in older computers since that's all they had.

CPUs are general purpose processor that do the general calculations and data. If a PC is handling AI algorithms, they are often done in the CPU. CPUs handle the more complex large file handling. CPUs handle the general interactions in a game. CPUs have fewer but massive complex processor cores optimized for large complex logic work.

GPUs are their own specialist computers optimized for complex graphics and physics vector calculations from hundreds of thousands of really tiny simple files. CPUs handle the general interactions in a game. GPUs handle the complex lighting, light rays, fragmentation physics and image 3D rendering. CUDA and Tensor Cores of a GPU are thousands of puny simple processors optimized for tiny random floating point calculations.

I think it’s more about the efficiency of doing the math than the ability to perform the math.

I don’t think thinking of them as “stripped down CPU cores” is accurate. They are very different beasts these days, with GPUs executing multiple threads in lockstep, not having a stack etc.

As far as “math”. GPUs can do pretty much everything in the sense they are turing complete, but at the same time if you’re asking instruction-wise, GPUs won’t have all the extra super-specific stuff that the x64 instruction set has for example. You’d need to write more stuff yourself.

I’m not sure exactly at what level you’re asking the question.

The most important thing about GPU cores is that they are parallel in nature. A lot of GPUs out there use 1024-bit arithmetic units that can process 32 numbers at the same time. That is, if you do something like a + b, both a and b are "vectors" consisting of 32 numbers. Since a GPU is built to process large amount of data simultaneously, for example shading all pixels in a triangle, this is optimal design that has good balance between cost, performance, and power consumption.

But the parallel design of GPU units also means that they will have problems if you want more execution granularity. For example in common control logic like "if condition is true do x, otherwise do y", especially if both x and y are complex things. Remember that GPUs really want to do the same thing for 32 items at a time, if you don't have that many things to work with, their efficiency will suffer. So a lot of common problem solutions that are formulated with "one value at a time" approach in mind won't translate directly to a GPU. For example, sorting. On a CPU it's easy to compare numbers and put them in sorted order. On a GPU you want to compare and order hundreds or even thousands of numbers simultaneously to get good performance, and it's much more difficult to design a program that will do it.

If you are talking about math specifically, well, it depends on the GPU. Modern GPUs are very well optimised for many operations and have native instructions to compute trigonometric functions (sin, cos), exponential functions and logarithms, as well as do complex bit manipulation. They also natively support a range of data values such as 32- and 16-bit floating point values. But 64-bit floating point value (double) support is usually lacking (either low performance or missing entirely).

what sort of math can cpus do that gpus can't

that's really restricting the problem space. Obviously (?) math wise it's the same thing: both of them can flip bits and arrange this bit flipping in ways useful for mathematics.

But CPUs are not just math. They always had I/O, almost always had interrupts, for many decades now they had protection rings and virtualization is now really common (Intel/AMD released support in 2005/2006). These are all supported in the instruction set.

Basic GPU handles algorithms for graphics and outputs to a display. Latest GPUs have specialized instructions to handle mathematical algorithms for parallelization so the Host OS hits certain routines it executes those on the GPU getting usually 8 - 32X speedup on execution over the CPU for mathematical routines.

Not so much GPUs themselves but GPU shader compilers really struggle on some code with a lot of branching and loops, operating on strings.

I think it’s more about the instruction set. CPUs are designed to be general purpose and can handle any processing required. GPUs have a specialized instruction set but are highly parallel which is useful for those specialized workloads.

Get 1 answer in 10 microseconds: Use a CPU.

Get 1000 answers in 1000 microseconds: use a GPU.

GPUs are not good with branching, as in taking different paths based on previous calculations.

A CPU core has a variety of arithmetic and logic circuits making it good for general purpose compute, whereas a GPU core (CUDA or Stream processor) has a much more limited set of logic units, which allows each one to be much smaller physically. This is why a GPU can fit thousands of cores while standard desktop CPUs are rarely more than 12.

The eli5 is that one is like a very strong man, capable of carrying large objects or pulling a car.

The other is like a man with a thousand arms. They might not be very strong, but they can carry a thousand plates.

The real difference between a GPU and a CPU is the access to the rest of the hardware. You could do logical operations on both, but a GPU can't address your hardware so it wouldn't be overall very useful as the only processing unit.

I imagine it like a construction site for a new house. Do you want 10 master craftsman who can do everything or 300 guys off the street who kinda know what they're doing and are really on there for grunt labor. For the tasks that require lots of manual labor you want the 300 guys, but for those tasks that aren't easy or aren't very parallel, you want fewer but more capable cores.

Think of it this way.

A CPU is a big digger, its bucket can dig a large chunk of earth with one scoop. Heavy lifting.

A GPU is a lot of small shovels, each digging a small amount of earth, but all at the same time.

So a CPU can do a large task at once, a GPU can do a lot of smaller tasks at once.

Which is great, the CPU is doing grunt work, the GPU is doing small calculations.

Both the GPU and the CPU being perfectly tasked for their specific purpose.

my guess is they just have different instruction sets but both are able to do basic add, substract and multiply

Security is an interesting reason that most people don't think about.

When you run a program on your computer, you're constantly swapping between user and privileged modes. For example, you don't want a website reading the stuff from your harddrive. Any such attempts must go to the OS which will then say the website doesn't have permission and refuse.

GPUs don't have a privileged mode. This isn't just because it wouldn't be useful. To the contrary, webGL or WebGPU have massive security issues because you are executing third-party code using drivers which themselves generally have root access. GPUs don't add such a mode because their hardware takes forever (in CPU terms) to get all the data loaded up and ready to execute. Moving to a different context every few microseconds would result in the GPU spending most of its time just waiting around for stuff to load up.

The solution to this problem would be decreasing latency, but all the stuff that does that for you is the same stuff that makes CPU cores hundreds of times larger than GPU cores. At that point, your GPU would just turn into an inferior CPU and an inferior GPU too.

It depends what type of GPU "core" you are talking about.

What NVIDIA refers to as CUDA/Tensor/RT cores are basically just glorified ALUs with their own itsy tiny control. But for the most part they are just an ALU.

For the most part CPUs tend to be more complete scalar processors, which they include the full control datapath as well as multiple Functional Units (FUs) not just an floating point unit.

The distictions are moot nowadays though; a modern GPU includes their own dedicated scalar core (usually in terms of a tiny ARM embedded core) for doing all the "housekeeping" stuff needed for them to interface with the outside world. And modern CPUs contain their own data-parallel functional units that can do some of the compute that GPUs can.

In the end the main difference is in terms of scale/width of data parallelism within a CPU (low) vs a GPU (high)

to echo others, it's not what, but how.

cpus do execution reordering and speculation to run one thread really fast. gpus have mostly avoided that and execute threads in large groups called "warps" (analogous to lanes of a SIMD unit).

Gpus may be Turing complete ( yes I just repeated that to annoy the correcting dude 😂). But the majority of normal workloads that do not require an insane amount of relatively simple parallel work would be excruciatingly slow on a gpu. The processing cores are small and slow ( but there are thousands of those).

• it does way less computations per clock
• it dossnt do branching very well which is the core ability of any cpu ( if then else in badly predictable ways)
• it has a slower clock speed
• it's cache lines are really bad unless you do a simple data in compute data out processing

Basically anything that is single threaded and moderately heavy in logic ( most OS core functionality and most application code) would be absolutely atrocious on a Cuda core. Also splitting out all parts that can be done on the Cuda cores would be a huge amount of work if your code is interspersed with the heavy stuff. That's why most computation are on cpus they are just much better at pretty much anything without much tuning unless you have a huge number crunching for example matrix computation. But if you have a couple million data points and you want to do some simple mathematics on them oooh the Cuda cores murder any cpu.

in a more practical way, CPU are way more efficient than GPU for lots tasks. for GPU once you move out of matrix multiplication you cannot do so much. a GPU will harvest a field faster, but will not efficiently cut a tree