Recently I had a request to configure the SMTP integraton with Monasca as part of the installation. Having already configured the SMTP server settings in the cloudConfig.yml file during the model building phase it should have been a simple case of configuring the notification details in the Operations Console or via the Monasca CLi. This didn’t work however, so here’s the steps I took to debug the SMTP integration:
Verify SMTP Server Communication
From each of the controllers run a simple curl email test to ensure the controllers can communicate with the email server as follows:
[Note: Ensure to amend the SMTP server and IP to match your requirements]
If this fails then the issue exists between the controller nodes and the SMTP server – verify that the SMTP details are correct,no firewalls blocking the ports etc. Check with the SMTP admin if they’re dropping the mails etc.
TCPDUMP and NETCAT (nc) are very useful utilities to verify this fundamental connectivity. Previous post here and all over the intertubes explain their use if required.
Use a Temporary ‘pop-up’ SMTP Server
If you don’t have an SMTP server or you wish to rule out you main SMTP server you could also follows these steps to quickly setup a testing SMTP server – you’ll see from the non standard ports I used above this is what I did initially.
I actually put my pop-up SMTP server on the deployer node [in a lab scenario – don’t do this in production] so I needed to open a port on the control-plane firewall to allow this the server to listen on port 3050 as follows:
So I’ve opened a range of ports here as I’ve other plans but you could simply open the single port required.
Once you’ve made these changes to the firewall cloud model file it’s the usual HLM process
Now copy the following file to the deployer node and save it as smtp.py Replace the ip address of the server and port you wish to use in the file.
Open a screen session (you may need to sudo apt-get install screen)
and run the script as follows:
Now you have your very own SMTP server to test the integration against.
With SMTP all verified we now need to configure Monasca.
Log in to the Helion Ops Console as a cloud admin user
Select the Notification Methods option from the ‘hamburger’ menu on the top left hand side of the screen.
Enter your the email address where you would like the emails to be send by creating a new email entry.
Now let’s create a test alert to see if we’re getting SMTP traffic.
Create a Test Alarm
Back to our ‘hamburger’ menu and this time select Alarm Creation and create a new alarm as follows – we want false alerts here – so I’ll use cpu.idle as the metric and set silly values –
Initially 2 Unknown Alarms may appear as this is the first time this alarm has run and it doesn’t have a previous state. However, be patient, in a few more minutes you should see the dashboard light up as shown below.
You can now select the alarms for more details.
And of course, what we’ve all been waiting for – email alerts!!!
[Notes : Troubleshooting Gotchas]
- If you’re still not seeing any traffic now AND your /var/log/monasca/notification/notification.log file is empty try re-configuring monasca
- If you have reused the Default email and the logfiles are complaining about an invalid ’email from’ address or the SMTP server requires a specific sender modify the default sender set in the /home/stack/helion/hos/ansible/roles/monasca-notification/defaults/main.yml . Ensure to do the full git commit…monasca-reconfigure cycle -identified above
- When reconfiguring existing alarm definitions ensure that you select BOTH the notifications checkbox AND the individual notification methods required.