A switch, or, more accurately, a toggle, moves an entity from one state to another until it is triggered again.
A button, or, more specifically, a gate, triggers something just once.
The original keyboard switch, the buckling spring, can be explained as a button, but what actually happens is you press a button, or a key, which compresses a spring, toggling a switch. It is in the release of the key that the distinction arrives.
Once you press the key and hold it, the system will register continuous input of that switch (toggle on). Once you release it, the toggle goes off again.
If the key were to function as a button, holding it would only register as one hit until it is pressed again.