SHR (hybrid RAID) doesn't really have a good open source alternative unless you're game to try something bleeding edge like seaweedfs.
The most recommended strategy to replace this is to use snapraid + mergerfs, which absolutely does work and scales but has it's own drawbacks, namely that it's a lot more management, it lacks bitrot protection like big players such as ZFS (though SHR also lacks this from my understanding), and it's offered protection is limited after files change.
In most cases I highly recommended biting the bullet and going with ZFS. It's a world class solution. With your mix of drives you'd end up using ZRAID-10, which uses more disk space but is flexible to upgrade and offers superior data protection and performance. There are a few push button solutions for ZFS including Proxmox and FreeNAS.
If safety isn't as important as storage space, stick with Xpenology.