Ostara, the name of the holiday we actually celebrated, is on the equinox, Easter stole a ton of traditions but slightly changed the day.
It's a celebration of spring, new life and fertility. Eggs and rabbits are both symbols of fertility hence their co-option into Easter. The history of Ostara herself is really interesting as you can actually trace her beginnings back Ishtar, one of humanity's easiest known deities.