I'm sending mails with PHP via a SMTP mail server. I have an SMIME Certificate for one email that sends all those mails (signed and also encrypted sometimes). This works perfectly.
Now I want to buy a (normal not SMIME) SSL Domain certificate (for the whole domain), is it possible to use this for my website and the signing / encrypting of those mails or do I need an SMIME certificate for the mails?
That won’t work for multiple reasons.
First, while it is technically possible to use an SSL certificate (or rather its private key) to sign or encrypt an email, a mail client won’t do it, because it requires that the certificate has been issued for email usage and for the sender’s email address.
Second, the recipient will have the same problem: their software won’t accept such a signature as valid.
Third, in order to encrypt an email, you need the recipient’s public key. Your own certificate doesn’t help you here.
You may want to look into DKIM to sign outgoing emails at the server level, but from your question, it’s not clear to me if that could help you.