And see, this right here is why Matrix is not ready for mainstream users. I've had the same exact issue. Key management with matrix rooms is hard as hell and really needs to be simplified. This is where something like simple X is better because it's a database, export, and the database file is encrypted and contains all the encrypted keys. So when you import the database with the password for the database, all the keys are there already.