Home Forums Support Invoice emails to clients marked spam and "Suspicious link" warning messages

This topic contains 27 replies, has 4 voices, and was last updated by  Hillel Coren 1 week, 3 days ago.

Viewing 28 posts - 1 through 28 (of 28 total)
  • Author
    Posts
  • #19603

    jmadrone
    Participant

    Environment:

    • Self-hosted
    • v4.5.10

    I am having a heck of a time with every email sent to real and test clients being marked as spam. Even worse, even after marking a message as ‘Not spam’ in Gmail, clicking the ‘Pay’ button in message body results in a popup window with warning message that says:

    Suspicious link
    This link leads to an untrusted site. Are you sure you want to proceed to invoice.mydomain.com?

    I am using a subdomain for my self-host install (eg. invoice.mydomain.com) and Gsuite (Google) SMTP servers to send mail. I have a user setup in Gsuite [email protected] that is used to send mail, so from and reply-to are the same.

    My domain has valid authentication with SPF, DKIM, and DMARC and receive a 9/10 score from mail-tester.com (the -1 point is for redirection found and url not formatted properly). I am using standard email templates with no additional html added.

    I have spent many hours reading many different topics about this and believe to have implemented every suggested fix and still no resolution. I don’t have any other issues with other emails whatsoever.

    Has anyone found a successful fix to this problem?

    #19605

    DavidBomba
    Keymaster

    Do you have https setup on your site? gmail throwing the error

    This link leads to an untrusted site. Are you sure you want to proceed to invoice.mydomain.com?

    would indicate the site is not secured by SSL, or the link is not being generated to the correct https:// link

    In regards to email delivery, this is not an issue with Invoice Ninja, this will be the way you have your email server configured there will be clues in the headers on the received email as to why it was marked as spam.

    #19606

    jmadrone
    Participant

    Hi and thanks for the quick reply.

    Yes I do have a valid cert and https is setup correctly. I understand that Invoice Ninja is just using the SMTP server that I define to send the messages. I do have one development since my post…

    Completely removing the email signature in settings, which did have a link to the client portal that was misspelled, has solved the suspicious link messages. I think that’s precisely what you were referring to in your message to me.

    I’m not sure exactly what’s causing the spam messages, but I have eliminated everything but the default templates with no additional text to see if that helps. I am sending test invoices to multiple email addresses I own and see that even within Gmail I get different results, depending on the security settings that are configured within Gsuite Admin Control Panel (not sure if Gmail users can set these). On one domain I have much looser settings, specifically I have turned off the two that relate to ‘Phishing’ emails, and invoices are coming to that inbox no problem. In another domain, with all security options enabled, the messages are identified specifically as “being like other phishing messages”.

    I will continue to troubleshoot tomorrow…

    #19607

    Hillel Coren
    Keymaster

    You may want to test with a different email provider and/or domain.

    #19608

    DavidBomba
    Keymaster

    @jmadrone

    have a look in the headers of a rejected email, all the information you need to debug this will be annotated in there.

    #19712

    jmadrone
    Participant

    DavidBomba can you tell me what I might be looking for in my email headers? I have since removed 100% of all links in my Email Signature and am using default templates with absolutely no additional anything. I am sending test emails to multiple addresses and providers and gmail is still being a real problem. Every invoice I send gets marked with as spam with suspected phishing warnings all over it and a red banner. That Suspicious Link warning message pops up no matter what you’re clicking on, so clicking the View Invoice button or even the domain.com link in the very bottom of the email that is not something I added but I’m guessing comes from my company information? I have downloaded the headers from several invoice emails and analyzed them with online tools as well as myself and don’t see anything out of the ordinary. I have checked my SSL cert with multiple online tools, which all checks out. I have used mxtollbox, mail-tester.com, Gsuite postmaster tools, etc and my DKIM, SPF, and DMARC all pass and pass acurately, meaning I have looked at each one and checked against what it should be. Thanks.

    #19713

    DavidBomba
    Keymaster

    @jmadrone

    What is your hosting platform?

    Are you hosting using a shared hosting platform, or are you running your own virtual machine?

    If the latter, can you confirm you have setup your hostname correctly on your machine, and also configured rdns to ensure google is not flagging your server as suspicious

    #19714

    jmadrone
    Participant

    I want to thank you for your quick response and continuing to assist with troubleshooting this. To answer your questions:

    1. Server is hosted with AWS EC2. IN is the only service running on this server… Nginx, MariaDB, and PHP 7.2
    2. Hostname = invoiceninja.mydoaim.com and was configured like this sudo hostnamectl set-hostname invoiceninja.mydomain.com
    3. rDNS = no, to be honest I don’t know much about rDNS except for what I’ve read in the previous 5 mins. Correct me if I’m wrong, but wouldn’t this be an issue if I was using this server as a mail server? Which it is not, as it is only using SMTP to send through Google’s mail servers.

    Thanks again

    #19715

    DavidBomba
    Keymaster

    @jmadrone

    From my understanding, even thou you are using gmail to send the mail, gmail will still look at the servers reputation that you are sending from, and it does mark down a server that may not be configured appropriately. As good practice I would always configure rDNS on any server.

    If this fails, let us know, you’ll probably need to send through the email headers for us to look into the issue further.

    #19768

    jmadrone
    Participant

    After reading the email headers multiple times and not seeing anything that jumps out I ran a search for the word spam and found this:

    X-CLX-Spam: false
    X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,,
    definitions=2019-02-27_14:,, signatures=0
    X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=54
    malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=214 mlxscore=0
    mlxlogscore=964 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
    engine=8.0.1-1812120000 definitions=main-1902270139

    I would be happy to pass along the headers, but they do have email addresses in them, so should I sanitize them and upload here, or send to you in an email, or something else?

    #19769

    jmadrone
    Participant

    Also interesting is that if I copy and paste all of the html in the email body of a message sent from IN server (AWS EC2 instance) that was marked as SPAM with all links contained therein SUSPICIOUS and send the same message from an the same email address via the same SMTP server using an email client (AirMail) the message is not marked SPAM and the links are not SUSPICIOUS.

    I have also discovered that the parts of the message header above are related to Apple iCloud email and Proofpoint MX machine learning spam filtering software that they must use as those parts of the header are only in test messages to an icloud email address. I also believe they have to do with the fact that I have email forwarding setup on that email account to automatically forward email sent there to my Gmail account.

    Sanitizing the full headers seems daunting, plus it makes it hard to follow/trace the various servers. Can I send to you directly or privately somehow?

    #19770

    DavidBomba
    Keymaster

    It sounds like AWS is filtering outbound mail?

    #19771

    jmadrone
    Participant

    I have added my server’s IP address to my SPF record so that it now reads (sanitized) v=spf1 ip1.2.3.4 include:_spf.google.com ~all , although this is still somewhat confusing but my understanding is that this is not really necessary at all as my server is not sending the mail as it is merely connecting to Google’s SMTP servers. This assertion is supported by the fact that all email headers, SPF record tools, Mail-Tester.com, etc. – only ever show the IP address of the Google server that actually sent the mail. I don’t believe that it hurts anything to include it though and so I have.

    I have also requested a PTR record for my AWS server using the form provided by Amazon online. This is a little different than the SPF record, but again my understanding is that my server (invoice.mydomain.com) is not actually sending the mail. Time to wait for this to propagate…

    #19772

    jmadrone
    Participant

    I think that is a logical place to look at this point DavidBomba. I am going to see if integrating Amazon SES helps in this matter. I have a Postmark account setup in Dev mode, but I really don’t want to spend the extra money at this point since I am already paying for multiple services such as Gsuite.

    I am going to wait for the PTR to fully propagate and then implement SES and then report back.

    #19788

    Mikhoul
    Participant

    I admit I did not read the whole thread but I was having a similar problem with my invoice being flagged as spam.

    So here’s my advice: If you use Google SMPT don’t forget to include _spf.google.com in the SPF: https://support.google.com/a/answer/33786

    Also don’t forget to install DMARC record it helped a lot for me: https://en.wikipedia.org/wiki/DMARC

    Here’s the tools I used to troubleshot my issues:

    I found your post and learned about https://www.mail-tester.com and it was the starting point for me to resolve my issues so it was helpful.

    Regards ☺️

    • This reply was modified 2 weeks, 5 days ago by  Mikhoul. Reason: typo
    • This reply was modified 2 weeks, 5 days ago by  Mikhoul.
    • This reply was modified 2 weeks, 5 days ago by  Mikhoul.
    #19797

    Hillel Coren
    Keymaster

    Thanks @mikhoul, that’s really helpful!

    #19805

    jmadrone
    Participant

    I have not found a solution to this problem. I have implemented SPF, DKIM, and DMARC, and have quadruple checked to ensure they are valid. I have a reverse DNS (PTR) record for my ec2 server so that the result of dig -x 1.2.3.4 returns something like this 4.3.2.1.in-addr.arpa. 299 IN PTR invoice.mydomain.com.,
    although I don’t believe that is an issue because the headers are the same as without and do not reference ec2 server’s IP as “sending” mail server anywhere. I did add my IP to the SPF record as well which doesn’t change anything, again because all the messages still are sent from smtp.gmail.com. My SPF now reads v=spf1 ip4:1.2.3.4 include:_spf.google.com ~all.

    As far as AWS being the culprit, I do not believe this to be the case because while they do filter port 25 and limit ability to send lots and lots of email from ports 465 & 587, I do not see anything in the headers to suggest any kind of machine learning/filtering being done, but I’m no expert in reading email headers so I could be wrong.

    As of today my messages to Gmail accounts are still being marked as Dangerous Spam, with links disabled and a giant RED Banner at the top of screen that says:

    This message seems dangerous
    Similar messages were used to steal people’s personal information. Avoid clicking links, downloading attachments, or replying with personal information.

    Then at the bottom of the screen, this message is displayed in orange banner:

    Downloading these attachments is disabled. This email has been identified as phishing. If you want to download these and you trust this message, click “Not spam” in the banner above.

    Google has several settings available to domain administrators which affect these messages. Depending on how the admin for the receiving domain has configured determines whether the messages make it through or not. The problem is that the default is “enabled” so I suspect most folks will have these “Safety” features on meaning they will not see my invoices/quotes/etc.

    These settings can be found here:
    Gsuite Admin > Apps > Settings for Gmail > Safety There are 3 settings:

    1. Attachments – Additional policies to protect against malware in emails.
    2. Links and external images – Additional settings to prevent email phishing due to links and external images.
    3. Spoofing and authentication – Additional settings to reduce phishing attacks due to spoofing and unauthenticated emails.

    Each of the 3 settings above has multiple sub-levels and options too. I have been going through them to find out which specific one is the culprit, but it is time consuming and so far results are not definitive as they seem to change. The only thing that is consistent is the message content. The way gmail handles them varies.

    I have been receiving DMARC reports from Postmark, which is great tool and feature they offer, and there does appear to be several “Unknown sources”, but there is nothing to be done about it according to Postmark’s FAQ and help guides.

    I am currently waiting for Amazon Support to remove email sending restrictions from my server, as part of getting Amazon SES going. However, their support tells me that the only limits imposed are on Port 25. Period. Regardless, I made the request and am jumping the hoops.

    How about that offer to look at some email headers?

    #19806

    jmadrone
    Participant

    It seems that the only way to prevent messages from going to spam is to contact each person using a personal email address you already communicate with them and request they look in their spam folder and add sender to their contacts, which doesn’t really seem like a good solution at all.

    I suppose I could tell IN to send all emails using said personal email address, which would be much more likely to be in receivers contacts. I don’t recall ever receiving an electronic invoice from [email protected] though and it seems a bit unprofessional. Right now I am using a separate Gsuite user called [email protected]. This user is a full user setup in the Gsuite Admin control panel. Argh…

    #19807

    DavidBomba
    Keymaster

    @jmadrone

    Do your domain names align?

    ie. The sending email user domain [email protected] is the same as the domain for the user embedded links? ie https://domain.com/view/invoice_url

    are the embedded links using https?

    #19814

    jmadrone
    Participant

    @davidbomba

    I’m not sure I completely understand about the domain names aligning, but my setup is like this:

    • My company uses tld mydomain.com
    • IN uses subdomain invoice.mydomain.com
    • IN SMTP user= [email protected]
    • the [email protected] user is a full/regular user (ie. not an alias) so the envelope and header FROM address/user does match (ie. smtp username = [email protected] & Reply-to = [email protected]).

      As far as links go from what I can tell the emails consist of:
      1. logo image – http://mydomain.com (assuming this url is populated from the Company Details?)
      – I currently have mydomain.com in Company Details > Website
      2. $viewButton – url is `https://invoice.mydomain.com/view/ajfeopiahf;ioah;jvn;ioaeh’
      3. mydomain.com link in footer – again, assuming this is populated from Company Details > Website ??

      I am using the Light Email Template
      —–

      When I copy & paste all message content into a new message and send to the same 4 test client email addresses none of this happens (ie. Not marked Spam + links open with no warnings about untrusted sites).

      Okay, so I would think that means that the content of the message is not causing problems. That makes me think AWS is filtering, but their support people have told me they are not, with the exception of port 25, which even that throttling has now been removed about an hour ago.

    #19816

    DavidBomba
    Keymaster

    So this may be the issue,

    You are sending from @domain.com but the view link is invoice.domain.com, this may be confusing Google / spam filters that you are impersonating @domain.com but sending users to invoice.domain.com

    I am sure this is why you are getting the phishing warnings.

    Send through an email with headers to [email protected] and i’ll have a look also, but I think we are on the right track now.

    #19842

    Mikhoul
    Participant

    You are sending from @domain.com but the view link is invoice.domain.com

    It should work this way since it’s the same domain, also I had the same setup with invoice.MyDomain.com sending from Mydomain.com wiout any problem.

    Another thing I can see is that your IP address are blacklisted on spam list, you cold check it here to see

    Here’s my result, since I use shared hosting I’m on 2 blacklist but right now it does not affect me but I work with my hosting company to be delisted from those list.

    Use this test to see if you are a major spam list and your IP is blacklisted.

    Have you tried all the tools/lockup from https://mxtoolbox.com/SuperTool.aspx ? πŸ€”

    Try them they are really powerful to find why emails are flagged as spam.

    Regards

    #19847

    jmadrone
    Participant

    I want to first thank everyone for their help and advice with this problem. I will admit I’ve learned a lot while troubleshooting this issue.

    I have identified the trigger for this which is in the email design/template selected in IN. If Light or Dark is selected, the messages are:

    • Marked as spam
    • not delivered to inbox
    • clicking any link, invluding view button shows “untrusted/suspicious” warning message, even after clicking ‘Not spam’ and ‘Report as not phishing’ from dropdown menu
    • If the Light theme is selected the messages are Not marked as spam, are delivered to the inbox, and clicking any/all links show no warnings of any kind.

      What’s strange is that copy/paste all content of either Light/Dark themes and re-sending from email client using same user and same SMTP settings/servers does not cause the spam and warning messages.

      I have used MXToolbox’s Supertool to check my domain and everything checks out. In the past week there have been times where my IP has been on 1 or 2 blacklists (out of many), but today that is not the case. This IP in question is not mine and belongs to Google as I am using their SMTP servers. I receive somewhere between an 8.5 and 9.5 out of 10 score on Mail-Tester.com. The negative points stem from things out of my control always and do not have anything to do with SPF DKIM or DMARC, as all of those are setup and verified to be valid.
      I am referring to mail-tester saying “your message could be improved” – I believe this refers to the low amount of text in the message.
      “2 broken links” – [591 - Error : Url not formatted properly] //fonts.googleapis.com/css?family=Open+Sans:300,700,900,100|Open+Sans:400,700,900,100 and
      “redirection found” – [302 - Redirection : Found] https://invoice.mydomain.com/view/keupajtjdtjvkf57bce4k404l7atnzdr

      I can go into my IN settings and switch back and forth between Themes and re-send invoices from each, and each time the messages are delivered according to the description/detail above. This is also not a browser caching issue because it doesn’t matter if new private windows are opened and the messages that are flagged stay flagged (ie. the warnings will still show after mail has come from the same address, marked not spam, etc., even days, weeks later – the same is true for Light themed mail that is delivered normally).

    #19848

    jmadrone
    Participant

    I’m sorry, I made a mistake above (can’t edit that one): Light/Dark themes = spam | Plain theme = not spam

    To follow up on that… So what’s different about the Light/Dark themes that could be causing this? Without getting into the HTML part yet, what jumps out at me is the display of a logo and a link in the footer of the email, which appears to come from the same place as the logo URL.

    Is this info coming from what is in the Settings > Company Details?

    I have Settings > System Settings > URL set to: https://invoice.mydomain.com and HTTPS Require is checked. In Settings > Company Details > Website I have: mydomain.com

    Also, if you click the question mark next to the Email Styles chooser in Settings > Email Settings a preview of each of the 3 styles is shown. The Plain emails show without a $viewButton and link only. In mine, Plain emails are still delivered with a $viewButton shown. The rest of emails look like preview. This leads me to further believe that it is not the redirection of the $viewButton (ie. negative points with mail-tester.com) and more likely has to do with the other links/logo image. Is that a fair assumption?

    • This reply was modified 2 weeks, 3 days ago by  jmadrone.
    • This reply was modified 2 weeks, 3 days ago by  jmadrone.
    #19852

    Mikhoul
    Participant

    One thing you could do to see exactly what trigger the spam filter is to copy the the source of the email (light of black) in another email sent with the same SMPT server that Invoice Ninja use and remove/modify slowly things in the source code to see exactly what is triggering the spam filter.

    Regards ☺️

    #19941

    Mikhoul
    Participant

    Here’s 2 more ressources I discovered to help to troubleshoot why email goes in the spam folder.

    https://glockapps.com/ is one of the best tool I used up to now but you have only 3 scan for free but you will know exactly which server put your mail in the spam folder after you have to pay.

    https://postmaster.google.com to help you to know why Google classify you email as spam.

    • This reply was modified 2 weeks, 1 day ago by  Mikhoul.
    • This reply was modified 2 weeks, 1 day ago by  Mikhoul.
    • This reply was modified 2 weeks, 1 day ago by  Mikhoul.
    #19999

    jmadrone
    Participant

    So it’s been a week and I would like to give an update for any others having this issue.

    I am happy to say that I have sent more than two dozen emails now with 100% success rate β€” no spam! However, I am forced to use the Plain email template only, so I am certainly not calling this a win or an acceptable solution.

    I have spent a great deal of time troubleshooting this issue and learned a lot. Since my last post I have successfully:

    • re-added an email signature with links
    • Changed invoice and reminder emails, including using many of the available variables (eg. $paymentButton etc.)
    • added an http link to myaccount.mydomain.com which redirects to my longer https://invoice.mydomain.com/client/login and not a single message has been marked as spam, no warnings about untrusted sites, or anything else

    I am still receiving the same score from Mail-Tester.com as I did on the very first failed message and my post here. My SPF and DKIM records are the same. I did add DMARC following the guide at Postmark and have been receiving weekly reports from them, but that did not change my score and other tools like MXToolbox and Google’s Postmaster Tools etc. all say the same thing they did from day one – PASS!

    If I go into IN settings and change the Email Theme to Light or Dark, then send an invoice using any template (ie. Initial, First Reminder…, etc.) to a test client I configured with 4 contacts, 3 of which are Gmail/Gsuite addresses, all 3 are flagged as spam and not delivered to inbox with all of the red flags, warnings, etc. discussed throughout this thread. Change the theme back to Plain and resend and boom β€” they land in inbox and clicking links and all the rest is good. At the same time I can say that it’s not a browser cache issue or anything like that because messages that were previously flagged remain flagged and show all warnings regardless of a message sent 2 minutes later being totally fine.

    I will continue to monitor and update… I have spent way too much time on this for now though and must just admit defeat and settle for not using the great looking themes, which is a major bummer πŸ™

    #20004

    Hillel Coren
    Keymaster

    That’s interesting to know, thanks for your help debugging!

    Maybe other users have used the app to spend spam with the themes causing filters to pick up on them?

Viewing 28 posts - 1 through 28 (of 28 total)

You must be logged in to reply to this topic.

Posted in: