are not audio drivers but PWM drivers
They can be both! A Class D audio amplifier can be constructed by rendering an audio signal into a PWM or PDM output signal, then passed through an RC filter to remove the switching noise, yielding only the intended audio.
That said, in this case, using the unfiltered PWM output would work for greeting cards, where audio fidelity is not exactly a high priority, but minimal parts count is.
This made me wonder if normal PWM controllers could be used to drive more power full LEDs.
What exactly did you have in mind as a "normal PWM controller"? There's a great variety of drivers that produce a PWM signal, some in the single watt category and some in the tens of kilowatts.
Whether they can drive "more powerful" LEDs is predominantly a function of the voltage and current requirements to fully illuminate the LEDs, plus what switching frequency range the LEDs can tolerate. Some LED modules that have built-in capacitors cannot be driven effectively using PWM, as well as anything which accepts AC rather than DC power. You'd need a triac to dim AC LED modules, and yet still, some designs simply won't dim properly.
My idea was to just remove the potentiometer and feed in music from Aux at that point.
You'll have to provide a schematic, as I'm not entirely sure where this potentiometer is. But be aware that the output current needed to drive a small speaker is probably insufficient to light up a sizable LED, nevermind the possibility of not even having enough voltage to meet the required forward voltage drop of the LED.
Is there a chance of this working?
It might, but only if everything just happens to line up. But otherwise, it's likely that it won't work as-is, due to insufficient drive current.
Another example of 120: the California State Legislature is a bicameral body with 120 total members, with 80 in the Assembly (lower house) and 40 in the Senate (upper house).
And a related piece of trivia regarding fractional currencies: although the USD has been decimalized more-or-less since its inception, the USA stock markets continued to use fractional prices for shares up until 2001, down to 1/16 of a dollar.