XenServer

Alerts

February 4, 2025

Contributed by:

You can configure XenServer to generate alerts based on any of the available host or VM metrics. In addition, XenServer provides preconfigured alerts that trigger when hosts undergo certain conditions and states.

View alerts

You can view these alerts using XenCenter or the xe CLI:

To view alerts in XenCenter, click Notifications and then Alerts.

The Alerts view displays various types of alerts, including performance alerts, system alerts, software update alerts, license alerts, and VM anti-affinity group alerts. For more information, see Alerts.
To view alerts using the xe CLI, type xe message-list.

You can narrow down the list of alerts by specifying additional parameters, for example:
- Specify the priority of the alerts you want to list: xe message-list priority=<priority>
- Specify whether it is a pool-related or VM-related alert: xe message-list class=<Pool|VM>
- Specify the name of the alert type: xe message-list name=<alert_name>. For a list of some of the available alert types, see System alerts.

You can also configure your environment to send alerts as an email. For more information, see Receive alerts through email.

Performance alerts

Performance alerts can be generated when one of the following values exceeds a specified threshold on a managed host, VM, or storage repository (SR): CPU usage, network usage, memory usage, control domain memory usage, storage throughput, or VM disk usage.

By default, the alert repeat interval is set to 60 minutes, it can be modified if necessary. Alerts are displayed on the Alerts page in the Notifications area in XenCenter. You can also configure XenCenter to send an email for any specified performance alerts along with other serious system alerts.

Any customized alerts that are configured using the xe CLI are also displayed on the Alerts page in XenCenter.

Each alert has a corresponding priority/severity level. You can modify these levels and optionally choose to receive an email when the alert is triggered. The default alert priority/severity is set at 3.

Priority	Name	Description	Default Email Alert
1	Critical	Act now or data may be permanently lost/corrupted.	Yes
2	Major	Act now or some services may fail.	Yes
3	Warning	Act now or a service may suffer.	Yes
4	Minor	Notice that something just improved.	No
5	Information	Day-to-day information (VM Start, Stop, Resume and so on)	No
?	Unknown	Unknown error	No

Configure performance alerts by using XenCenter

In the Resources pane, select the relevant host, VM, or SR, then click the General tab and then Properties.

Select the Alerts tab. The following table summarizes which alerts are available for hosts, VMs, or SRs:

Alert name	Host	VM	SR	Description
Generate CPU usage alerts	X	X		Set the CPU usage and time threshold that trigger the alert.
Generate control domain CPU usage alerts	X			Set the control domain CPU usage and time threshold that trigger the alert.
Generate memory usage alerts	X			Set the memory usage and time threshold that trigger the alert.
Generate control domain memory usage alerts	X			Set the control domain memory usage and time threshold that trigger the alert.
Generate control domain free memory alerts	X			Set the control domain free memory and time threshold that trigger the alert.
Generate disk usage alerts		X		Set the disk usage and time threshold trigger the alert.
Generate storage throughput alerts			X	Set the storage throughput and time threshold that trigger the alert. Note: Physical Block Devices (PBD) represent the interface between a specific XenServer host and an attached SR. When the total read/write SR throughput activity on a PBD exceeds the threshold you have specified, alerts are generated on the host connected to the PBD. Unlike other XenServer host alerts, this alert must be configured on the SR.
Generate network usage alerts	X	X		Set the network usage and time threshold that trigger the alert.

To change the alert repeat interval, enter the number of minutes in the Alert repeat interval box. When an alert threshold has been reached and an alert generated, another alert is not generated until after the alert repeat interval has elapsed.

Click OK to save your changes.

For comprehensive details on how to view, filter and configure severities for performance alerts, see Configuring Performance Alerts in the XenCenter documentation.

Configure performance alerts by using the xe CLI

Note:

Triggers for alerts are checked at a minimum interval of five minutes. This interval avoids placing excessive load on the system to check for these conditions and reporting of false positives. Setting an alert repeat interval smaller than five minutes results in the alerts still being generated at the five minute minimum interval.

The performance monitoring perfmon tool runs once every five minutes and requests updates from XenServer which are averages over one minute. These defaults can be changed in /etc/sysconfig/perfmon.

The perfmon tool reads updates every five minutes of performance variables running on the same host. These variables are separated into one group relating to the host itself, and a group for each VM running on that host. For each VM and host, perfmon reads the parameter other-config:perfmon and uses this string to determine which variables to monitor, and under which circumstances to generate a message.

For example, the following shows an example of configuring a VM “CPU usage” alert by writing an XML string into the parameter other-config:perfmon:

xe vm-param-set uuid=vm_uuid other-config:perfmon=\

'<config>
    <variable>
        <name value="cpu_usage"/>
        <alarm_trigger_level value="0.5"/>
    </variable>
</config>'
<!--NeedCopy-->

Note:

You can use multiple variable nodes.

After setting the new configuration, use the following command to refresh perfmon for each host:

xe host-call-plugin host=host_uuid plugin=perfmon fn=refresh
<!--NeedCopy-->

If this refresh is not done, there is a delay before the new configuration takes effect, since by default, perfmon checks for new configuration every 30 minutes. This default can be changed in /etc/sysconfig/perfmon.

Valid VM elements

name: The name of the variable (no default). If the name value is either cpu_usage, network_usage, or disk_usage, the rrd_regex and alarm_trigger_sense parameters are not required as defaults for these values are used.
alarm_priority: The priority of the alerts generated (default 3).
alarm_trigger_level: The level of value that triggers an alert (no default).
alarm_trigger_sense: The value is high if alarm_trigger_level is a maximum value otherwise low if the alarm_trigger_level is a minimum value (the default high).
alarm_trigger_period: The number of seconds that values (above or below the alert threshold) can be received before an alert is sent (the default is 60).
alarm_auto_inhibit_period: The number of seconds this alert will be disabled after an alert is sent (the default is 3600).
consolidation_fn: Combines variables from rrd_updates into one value. For cpu-usage the default is average, for fs_usage the default isget_percent_fs_usage and for all others - sum.
rrd_regex: Matches the names of variables from xe vm-data-sources-list uuid=vm_uuid, to compute performance values. This parameter has defaults for the named variables:
- cpu_usage
- memory_internal_free
- network_usage
- disk_usage

If specified, the values of all items returned by xe vm-data-source-list whose names match the specified regular expression are consolidated using the method specified as the consolidation_fn.

Valid host elements

name: The name of the variable (no default).
alarm_priority: The priority of the alerts generated (default 3).
alarm_trigger_level: The level of value that triggers an alert (no default).
alarm_trigger_sense: The value is high when alarm_trigger_level is a maximum value otherwise low if the alarm_trigger_level is a minimum value. (default high)
alarm_trigger_period: The number of seconds that values (above or below the alert threshold) can be received before an alert is sent (default 60).
alarm_auto_inhibit_period: The number of seconds that the alert is disabled for after an alert is sent. (default 3600).
consolidation_fn: Combines variables from rrd_updates into one value (default sum - or average)
rrd_regex: A regular expression to match the names of variables returned by the xe vm-data-source-list uuid=vm_uuid command to use to compute the statistical value. This parameter has defaults for the following named variables:
- cpu_usage
- network_usage
- memory_free_kib
- sr_io_throughput_total_xxxxxxxx (where xxxxxxxxis the first eight characters of the SR-UUID).

SR Throughput: Storage throughput alerts must be configured on the SR rather than the host. For example:

xe sr-param-set uuid=sr_uuid other-config:perfmon=\
'<config>
    <variable>
        <name value="sr_io_throughput_total_per_host"/>
        <alarm_trigger_level value="0.01"/>
    </variable>
</config>'
<!--NeedCopy-->

Generic example configuration

The following example shows a generic configuration:

<config>
    <variable>
    <name value="NAME_CHOSEN_BY_USER"/>
    <alarm_trigger_level value="THRESHOLD_LEVEL_FOR_ALERT"/>
    <alarm_trigger_period value="RAISE_ALERT_AFTER_THIS_MANY_SECONDS_OF_BAD_VALUES"/>
    <alarm_priority value="PRIORITY_LEVEL"/>
    <alarm_trigger_sense value="HIGH_OR_LOW"/>
    <alarm_auto_inhibit_period value="MINIMUM_TIME_BETWEEN_ALERT_FROM_THIS_MONITOR"/>
    <consolidation_fn value="FUNCTION_FOR_COMBINING_VALUES"/>
    <rrd_regex value="REGULAR_EXPRESSION_TO_CHOOSE_DATASOURCE_METRIC"/>
    </variable>

    <variable>
    ...
    </variable>

    ...
</config>
<!--NeedCopy-->

System alerts

The following table displays the system events/conditions that trigger an alert to be displayed on the Alerts page in XenCenter.

Name	Priority/Severity	Description
license_expires_soon	2	XenServer License agreement expires soon.
ha-statefile_lost	2	Lost contact with the high availability Storage Repository, act soon.
ha-heartbeat_approaching_timeout	5	High availability approaching timeout, host may reboot unless action is taken.
ha_statefile_approaching_timeout	5	High availability approaching timeout, host may reboot unless action is taken.
haxapi_healthcheck_approaching_timeout	5	High availability approaching timeout, host may reboot unless action is taken.
ha_network_bonding_error	3	Potential service loss. Loss of network that sends high availability heartbeat.
ha_pool_overcommited	3	Potential service loss. High availability is unable to guarantee protection for configured VMs.
ha_poor_drop_in_plan_exists_for	3	High availability coverage has dropped, more likely to fail, no loss present yet.
ha_protected_vm_restart_failed	2	Service Loss. High availability was unable to restart a protected VM.
ha_host_failed	3	High availability detected that a host failed.
ha_host_was_fenced	4	High availability rebooted a host to protect against VM corruption.
redo_log_healthy	4	The XAPI redo log has recovered from a previous error.
redo_log_broken	3	The XAPI redo log has encountered an error.
ip_configured_pif_can_unplug	3	An IP configured NIC can be unplugged by XAPI when using high availability, possibly leading to high availability failure.
host_sync_data_failed	3	Failed to synchronize XenServer performance statistics.
host_clock_skew_detected	3	The host clock is not synchronized with other hosts in the pool.
host_clock_went_backwards	1	The host clock is corrupted.
pool_master_transition	4	A new host has been specified as pool coordinator.
pbd_plug_failed_on_server_start	3	The host failed to connect to Storage at boot time.
auth_external_init_failed	2	The host failed to enable external AD authentication.
auth_external_pool_non-homogeneous	2	Hosts in a pool have different AD authentication configuration.
multipath_period_alert	3	A path to an SR has failed or recovered.
bond-status-changed	3	A link in a bond has disconnected or reconnected.

Software update alerts

XenCenter old: XenServer expects a newer version but can still connect to the current version
XenCenter out of date: XenCenter is too old to connect to XenServer
XenServer out of date: XenServer is an old version that the current XenCenter cannot connect to
License expired alert: XenServer license has expired
Missing IQN alert: XenServer uses iSCSI storage but the host IQN is blank
Duplicate IQN alert: XenServer uses iSCSI storage, and there are duplicate host IQNs

Receive alerts through email

You can configure XenServer to send email notifications when XenServer hosts generate alerts. The mail-alarm utility in XenServer uses sSMTP to send these email notifications. You can enable basic email alerts by using XenCenter or the xe Command Line Interface (CLI). For further configuration of email alerts, you can modify the mail-alarm.conf configuration file.

Use an SMTP server that does not require authentication. Emails sent through SMTP servers that require authentication cannot be delivered.

Enable email alerts by using XenCenter

In the Resources pane, right-click on a pool and select Properties.
In the Properties window, select Email Options.
Select the Send email alert notifications check box. Enter your preferred destination address for the notification emails and SMTP server details.
Choose your preferred language from the Mail language list. The default language for performance alert emails is English.

Enable email alerts by using the xe CLI

To configure email alerts, specify your preferred destination address for the notification emails and SMTP server:

xe pool-param-set uuid=pool_uuid other-config:mail-destination=joe.bloggs@example.com
xe pool-param-set uuid=pool_uuid other-config:ssmtp-mailhub=smtp.example.com:<port>
<!--NeedCopy-->

XenServer automatically configures the sender address as noreply@<hostname>. However, you can set the sender address explicitly:

xe pool-param-set uuid=pool_uuid other-config:mail-sender=serveralerts@example.com
<!--NeedCopy-->

When you turn on email notifications, you receive an email notification when an alert with a priority of 3 or higher is generated. Therefore, the default minimum priority level is 3. You can change this default with the following command:

xe pool-param-set uuid=pool_uuid other-config:mail-min-priority=level
<!--NeedCopy-->

Note:

Some SMTP servers only forward mails with addresses that use FQDNs. If you find that emails are not being forwarded it might be for this reason. In which case, you can set the server host name to the FQDN so this address is used when connecting to your mail server.

To configure the language for the performance alert emails:

xe pool-param-set uuid=pool_uuid other-config:mail-language=ja-JP
<!--NeedCopy-->

The default language for performance alert emails is English.

Further configuration

To further configure the mail-alarm utility in XenServer, create an /etc/mail-alarm.conf file containing the following:

root=postmaster
authUser=<username>
authPass=<password>
mailhub=@MAILHUB@
<!--NeedCopy-->

/etc/mail-alarm.conf is a user-supplied template for sSMTP’s configuration file ssmtp.conf and is used for all alerts generated by XenServer hosts. It consists of keys where key=@KEY@ and @KEY@ is replaced by the corresponding value of ssmtp-key in pool.other_config. These values are then passed to ssmtp, allowing you to control aspects of the sSMTP configuration using values from pool.other_config. Note how @KEY@ (uppercase) corresponds to ssmtp-key (lowercase, prefixed by ssmtp-).

For example, if you set the SMTP server:

xe pool-param-set uuid=pool_uuid other-config:ssmtp-mailhub=smtp.example.com
<!--NeedCopy-->

and then add the following to your /etc/mail-alarm.conf file:

mailhub=@MAILHUB@
<!--NeedCopy-->

mailhub=@MAILHUB@ becomes mailhub=smtp.example.com.

Each SMTP server can differ slightly in its setup and may require extra configuration. To further configure sSMTP, modify its configuration file ssmtp.conf. By storing relevant keys in the mail-alarm.conf file, you can use the values in pool.other_config to configure sSMTP. The following extract from the ssmtp.conf man page shows the correct syntax and available options:

NAME
    ssmtp.conf – ssmtp configuration file

DESCRIPTION
    ssmtp reads configuration data from /etc/ssmtp/ssmtp.conf The file con-
    tains keyword-argument pairs, one per line. Lines starting with '#'
    and empty lines are interpreted as comments.

The possible keywords and their meanings are as follows (both are case-
insensitive):

    Root
    The user that gets all mail for userids less than 1000. If blank,
    address rewriting is disabled.

    Mailhub
        The host to send mail to, in the form host | IP_addr port :
        <port>. The default port is 25.

    RewriteDomain
    The domain from which mail seems to come. For user authentication.

    Hostname
        The full qualified name of the host. If not specified, the host
        is queried for its hostname.

    FromLineOverride
        Specifies whether the From header of an email, if any, may over-
        ride the default domain. The default is "no".

    UseTLS
    Specifies whether ssmtp uses TLS to talk to the SMTP server.
    The default is "no".

    UseSTARTTLS
        Specifies whether ssmtp does a EHLO/STARTTLS before starting TLS
        negotiation. See RFC 2487.

    TLSCert
        The file name of an RSA certificate to use for TLS, if required.

    AuthUser
        The user name to use for SMTP AUTH. The default is blank, in
        which case SMTP AUTH is not used.

    AuthPass
        The password to use for SMTP AUTH.

    AuthMethod
        The authorization method to use. If unset, plain text is used.
        May also be set to "cram-md5".
<!--NeedCopy-->

The official version of this content is in English. Some of the Cloud Software Group documentation content is machine translated for your convenience only. Cloud Software Group has no control over machine-translated content, which may contain errors, inaccuracies or unsuitable language. No warranty of any kind, either expressed or implied, is made as to the accuracy, reliability, suitability, or correctness of any translations made from the English original into any other language, or that your Cloud Software Group product or service conforms to any machine translated content, and any warranty provided under the applicable end user license agreement or terms of service, or any other agreement with Cloud Software Group, that the product or service conforms with any documentation shall not apply to the extent that such documentation has been machine translated. Cloud Software Group will not be held responsible for any damage or issues that may arise from using machine-translated content.

Was this helpful

Alerts

February 4, 2025

Contributed by:

February 4, 2025

Contributed by:

Alerts

View alerts

Performance alerts

Configure performance alerts by using XenCenter

Configure performance alerts by using the xe CLI

Valid VM elements

Valid host elements

Generic example configuration

System alerts

Software update alerts

Receive alerts through email

Enable email alerts by using XenCenter

Enable email alerts by using the xe CLI

Further configuration

In this article