The "To" address in direct 1:1 emails sent from the contact record reflects the associated email address in the contact record. This address cannot be modified without changing the address in the contact record itself.
The "From" address in a direct 1:1 email will always reflect the email address associated with the logged-in user. This can be changed by changing the user's email address under Settings > Users if contacts need to reach out in a personal email from a different address.