It's so hard!

It's really hard! But it's really rewarding too. And as a computing/music student [1], you're in a great major to start!

First off, if you just want to make your own effects and you're not really interested in distributing them or making them public, I recommend using JSFX. It's way easier. You can read through the entire spec in a night. JSFX support is built into REAPER, and apparently YSFX allows you to load JSFX code into other DAWs, although I haven't tested it. JSFX plugins are compiled on the fly (unlike VST plugins, which are compiled ahead of time and distributed as DLLs), so you just write them up as text files.

However, their capabilities are limited compared to VST, AU, LV2, AAX [2], and other similar plugin formats. Also, pre-compiled plugins perform better. That's why plugins are released as such.

So if you plan on writing pre-compiled plugins for public consumption, you'll need to do some C++ programming.

IMO the most important thing to learn for plugin design is how to code well, particularly in C++ with Git and JUCE.

If you learn how to code with good practices, you can compensate for all other deficiencies.

Between "music", "engineering", and "software development", plugin design feels the most like "software development".

99.9% of all plugins are written in C++, and most of those are done (both proprietary and FOSS) with the JUCE library. School taught me the basics of C++ but they don't teach you how to code well. Particularly, your DSP code needs to meet a soft real-time constraint. You have to use multithreading because you have a thread for the audio signal (which must NEVER get interrupted) and at least one thread for the GUI.

You also need to figure out which parts of the C++ standard library are real-time safe, and which aren't. Here's a good talk on that.

If you use JUCE or a similar development library then they have well-tested basic DSP functions, meaning you can get by without doing all the math from scratch.

Start watching Audio Developer Conference talks like TV as they come out. JUCE has a tutorial, and MatKat released a video tutorial guiding the viewer through coding a simple EQ plugin [3]. JUCE plugins are basically cross platform, and can typically be compiled as VSTs on Windows, AU plugins on Mac, and LV2 plugins on Linux.

JUCE is a really complicated library even though it vastly simplifies the process (because audio plugin development is inherently hard!). You're going to have to learn to read a LOT of documentation and code.

I also recommend learning as much math as you can stomach. Start with linear algebra, calculus, Fourier analysis, circuit theory, and numerical analysis (especially Padé approximants), in that order. Eventually, you'll want to roll your own math, or at least do something that JUCE doesn't provide out the box. Julius O Smith has some really good free online books on filters, Fourier Analysis, and DSP with a music focus.

If you're willing to ~~sail the high seas to LibGen~~ buy a book, I recommend Digital Audio Signal Processing by Udo Zolzer for "generic" audio signal processing, and DAFX: Digital Audio Effects by Zolzer for coverage of nonlinear effects, which are typically absent from DSP engineering books. I also recommend keeping a copy of Digital Signal Processing by Proakis and Manolakis on hand because of its detailed coverage of DSP fundamentals, particularly the coverage of filter structures, numerical errors, multirate signal processing, and the Z transform.

A little bit of knowledge about machine learning and optimization is good too, because sometimes you need to solve an optimization problem to synthesize a filter, or possibly in a fixed time as your actual output (example: pitch shifting). Deep learning is yielding some seriously magical effects, so I do recommend you learn it at your own pace.

DSP basically requires all the math ever, especially the kind of DSP that we want to do as musicians, so the more you have the better you'll be.

[1] IMO that would have been the perfect major for me, that or acoustical engineering, if anything like that existed in my area when I went to recording school 10 years ago. While my recording degree taught me some really valuable stuff, I kinda wish that they pushed us harder into programming, computing, and electronics.

[2] AAX requires you to pay Avid to develop. So I never use AAX plugins, and I have no intention of supporting the format once I start releasing plugins for public consumption, despite its other technical merits.

[3] Over half of MatKat's tutorial is dedicated towards GUI design, i.e. the audio part is basically done but the interface looks boring and default. GUI design and how your GUI (editor component) interacts with the audio processor component are extremely important and time-consuming parts of plugin design. Frankly, GUI design has been by far the most complicated thing to "pick up", and it's why I haven't released anything yet.