I finally figured this out searching in the CodeIgniter forum.
In /opt/ASTPP/web_interface/astpp/system/libraries/Email.php there is a setting on about line 41
var $smtp_crypto = " "; // SMTP Encryption. Can be null, tls or ssl.
Since my email server is using STARTTLS, I needed to put “tls” in the quotes, and made sure the account settings in the GUI is using port 587.
Also to @devangn:
Can the documentation be put on some sort of thing like a wiki where others can contribute? Issues like this is nowhere to be found in your documentation, and others cannot help with it.