Protecting XenServer Networks and Storage

In a XenServer environment, networking occurs not only on the physical level but also the logical level. From a network Layer 2 perspective, XenServer functions as a virtual network switch that can send traffic internally within the host (such as for private networks) and externally to the physical network.

Consequently, it is important that the XenServer environment is secured at all its levels: the virtual machine (guest) layer, the hypervisor layer, and the physical network layer, including storage networks.

Recommendation

We recommend that you ensure that there is no data that crosses between networks, including the logical networks in the hypervisor layer.

Network Traffic Separation

XenServer has three distinct categories of network traffic in each pool: (a) management traffic, (b) storage traffic, and (c) VM (guest operating system) traffic. You can configure your XenServer hosts to send this traffic over separate or shared networks.

The following illustration provides an example of this separation where different classes of traffic use different physical NICs.

This illustration shows how NICs that are not designated for management or storage traffic only carry VM traffic.

• Management Traffic: XenServer uses the management network for XenServer management communications, including traffic between hosts, live migration, VM import and export, and resource pool metadata backups. The management network is the network connected to the NIC that is configured as the management interface. Management traffic does not include communications between guest virtual machines but does include traffic involving XenCenter, the administration console.

  XenServer installation creates the management network on each host and assigns it to a specific NIC that will function as the management interface. XenServer will allow you to use a single interface for all three types of traffic: management, storage and guest. However, this is not the most secure networking configuration, and we strongly recommend separating traffic as described throughout this section.

• Storage Traffic: The term storage traffic refers to Fibre Channel traffic and IP-based storage traffic, such as iSCSI software initiator traffic, NFS traffic, and SMBv3 traffic. However, when this guide discusses placing storage traffic on its own network this specifically refers to iSCSI and NFS traffic. (Fibre Channel traffic is inherently on its own network.) Storage traffic is discussed in more detail throughout this section and in Protecting Virtualized Storage.

• Guest Traffic: Guest traffic refers to traffic being sent to or from guest virtual machines.

  Physical network separation provides additional security. The following illustration provides an example.

This illustration shows the management interface, the NIC used for management communications, and the management network. The management network is separated from the storage network and the guest network.

Recommendation:

We strongly recommend physically separating the management, storage, and guest networks through NICs, cables, switches, ports and network configuration as well as logically separating through configuration in XenServer. This includes ensuring that you do not unintentionally send guest traffic across the management network, as explained in the section that follows.

Ensuring Traffic is Separated

To understand how to separate traffic, it is important to understand how VMs communicate.

A VM can only use a NIC for guest traffic if it has a virtual network interface on the same network as the NIC.

Not all NICs have virtual network interfaces associated with them. If you do not configure a virtual network interface connecting to the management network, the management NIC becomes dedicated for management traffic. For example, in the previous illustration, there are NICs connected to the management and storage networks that do not have corresponding virtual network interfaces.

The previous illustration depicts VMs that only have virtual network interfaces for the guest network. The VMs are not connected to the management or storage network: the VMs do not have virtual network interfaces that connect to those networks.

Recommendation:

We recommend that you isolate production networks from staging, testing and development networks.

Planning a Separate Storage Network

Recommendation

We recommend that you segregate traffic by configuring an additional storage interface(s) for IP-based storage traffic and configuring XenServer to access storage through that interface. (However, you must also then physically isolate the guest networks and configure virtual machines only to use those isolated networks). Note that additional interfaces are also known as secondary interfaces in some literature.

The overall process for creating a separate storage network is as follows:

1. Configuring physical network infrastructure so that different traffic is on different subnets.
2. Creating a storage interface to use the new network.

Consider configuring redundancy, either multipathing or bonding, during the above steps if desired.

Following the above process lets you establish separate networks for IP-based traffic provided that:

• You do not configure XenServer to use this network for any other purpose (for example, by connecting a virtual network interface to this network).
• The appropriate physical network configuration is in place.

For example, to dedicate a NIC to storage traffic, the NIC, storage target, switch, and/or VLAN must be configured so that the target is only accessible over the assigned NIC.

To ensure that the storage traffic is separated from the management traffic, the storage network must be on a different subnet. The subnet for storage must be a separate IP subnet that is not “routable” from the management interface. If the physical or logical configuration does not enforce the traffic separation, then XenServer may direct storage traffic over the management interface after a host reboot, due to the order in which XenServer initializes NICs.

Planning a Separate Guest Network

Recommendation

• We recommend that you configure all VMs to have guest networks only. VMs do not require access to storage and management networks as explained below.

• We recommend that you configure your guest networks correctly so they do not unintentionally send traffic over the management and storage networks:
  • Ensure the guest network is physically isolated from management and storage traffic. Connect the NIC that the VMs will use for guest traffic to a switch port that will be used for VM traffic only. This port must be physically isolated from the storage and management networks.
  • Create a virtual network interface on the same network(s) as the NIC.
  • Do not route any types of traffic, other than guest traffic, across that NIC.
  • When you create VMs, if you configured the management interface on NIC 0, make sure that NIC 0 is not enabled in the VM wizard when you create the VM, as explained in the procedure that follows.

• Alternative: In situations where hosts only have one NIC dedicated to guest traffic, we recommend separating traffic into separate logical networks. For example, a common configuration is for an application server to manage “front-end” traffic (for example, to a web server) and “back-end” traffic (for example, to a database). This can be done using VLANs. A VLAN tags the Ethernet traffic separately but traffic still goes over the same physical NIC on the host.

• We recommend that you do not assign the control domain an IP address for any network other than storage networks and the isolated management network. Do not place guest VM traffic on any network that has a control domain IP address.
• We recommend that, if you have a guest network that connects to a network that is not under your organizational control (such as the Internet), you make sure the local segment of the network is protected. Any unprotected network may be vulnerable to Level 2 network attacks, such as fake DHCP attacks, that must be performed from the same switch as the attack’s target.
• We recommend that, if you have some VMs that require Internet access and some that do not, you configure separate networks so that the VMs that do not require Internet access are not directly exposed to possible attacks from the Internet.

To configure VMs with guest networks only

Note:

This procedure may be easier to perform if you change the names of the networks to assign them clear names as described in Naming Networks Clearly.

1. After launching the XenCenter New VM wizard (VM > New VM), follow the prompts until you reach the Networking page.

2. On the Networking page, remove all networks that the VM will not be using. Select the network to remove and click Delete.

   

   By default, XenServer installation configures the management interface on Network 0, which is typically NIC0.

   The resulting networks may look something like the following:

   

3. If necessary, click Properties to configure QoS settings or Virtual MAC addresses.

   Important:

   Any virtual network interfaces that appear in this dialog box will be created as networks on the virtual machine.

Naming Networks Clearly

Recommendation

We recommend that you rename the default networks so they have clear names that are meaningful to your configuration to prevent inadvertently connecting a guest to a network to which it should not have access, such as the management network.

For example, change the default names XenServer assigns to NICs to ones that represent the functions they perform in your environment:

To change the names of networks

In the Networking tab in XenCenter, select the network and click Properties, as shown in the following screen capture:

This screen capture displays the names of networks, which were changed from their default names for clarity. You can also add information about the network to the description column.

You can also change network names by running the following command:

# xe network-param-set uuid=<network uuid> name-label="<new network name>"
<!--NeedCopy-->

For example, to assign Network 1 the name “Storage Network” run:

# xe network-param-set uuid=213e396e-1e1f-0744-7a43-64c3e9dfd649 \ name-label="Storage Network"
<!--NeedCopy-->

Using the name-label parameter is the same as renaming the network in XenCenter.

Tip:

To obtain the network UUID, run xe network-list.

Protecting the Management Network

Protecting the management network is important because this network can potentially provide access to all components connected to a host. Communication between XenServer 8 hosts and both XenCenter and other XenServer 8 hosts is over (encrypted) TLS connections to port 443. However, XenServer management functions listen on both port 80 (unencrypted) and port 443 (TLS encrypted) for management API requests. This allows interoperability with older versions of the product and other (e.g. third-party) products.

Recommendation

Unless you have third-party management API agents that are unable to use TLS, we recommend that you block access to port 80 with the CLI command: xe pool-param-set uuid=<pool uuid> https-only=true.

If you need to reverse this command, for example if you need to migrate VMs to a XenServer 8 host from an earlier version of the product, you can do so with the CLI command: xe pool-param-set uuid=<pool uuid> https-only=false.

It is possible to configure the management interface so that an IP address is not bound to it. However, this means that none of the management functions will work from outside the local console on the XenServer host. Be aware that in this configuration you cannot create resource pools, import/export VMs, or otherwise take advantage of features such as email alerting.

Configuring Switch Port Locking

In environments with untrusted guest VMs, it may be desirable to use security measures, such as spoofing protection, to reduce the ability of VMs to attack other virtual machines over the network.

Recommendation

• We recommend that environments with potentially hostile or unknown guest traffic configure the XenServer switch-port locking feature:
• We recommend using the switch-port locking feature to define specific IP addresses from which an individual VM is allowed to send traffic.
• The XenServer switch-port locking feature lets you control traffic being sent from unknown, untrusted, or potentially hostile VMs by limiting their ability to pretend they have a MAC or IP address that was not assigned to them.
• We recommend that environments that have both (a) VMs containing sensitive data and (b) untrusted VMs use the switch-port locking feature to prevent an untrusted VM from sending spoofed traffic to sensitive VMs.

The XenServer switch-port locking feature may assist in deployments that have a network architecture in which each VM has a public, Internet-connected IP address. For more information about switch-port locking, see the product documentation.

Securing the Physical Network

Recommendation

• We recommend that you take precautions to ensure you do not inadvertently cable the management interface into the guest network. For example:
  • We recommend consistently designating the same NICs for use with the same networks across all hosts in all pools.
  • In environments with multiple pools, We recommend always designating specific ports for specific types of traffic and using these ports consistently. For example, across all pools always use the first network port for your management network, the second network port for your storage network, and so on.
  • Where your organization does not already have cabling standards, We recommend that you consider labelling or color-coding cables to indicate their purpose.

• We recommend that you erase all previous configurations when you reuse physical switches.

  Previous passwords or configurations could potentially expose your environment to hostile entities or traffic. For example, somebody might have configured the switch to route traffic automatically to another site or network.

• We recommend that you protect the physical location of the network, including the security of the physical area where the network is located.

Protecting Virtualized Storage

Data storage is considered to be virtualized when there is not a one-to-one relationship between the physical storage device and the resulting virtualized storage devices (or disks). For example, when one physical storage device is presented to users as one or more virtualized storage devices or conversely when multiple physical devices are presented as one virtual disk. While virtualizing data storage reduces the user’s perception of complexity, it is important to ensure storage is virtualized securely.

For companies who want to secure and monitor their virtualized storage, it is worth noting that virtualized storage can reside in multiple physical locations simultaneously.

Recommendation

• We recommend that you consider where all the physical drives being virtualized are located when determining the physical security of disks and you do not place disks in a location where they are physically at risk.
• For example, if you add another SR because your original SR ran out of space and the new SR is in a different physical location (for example, on a server that might not be in the same machine room), ensure both locations are physically secure.

Dedicated storage NICs require an IP address that must be on a different IP subnet to the main administration interface. You can use XenCenter to dedicate a storage NIC and also to bond multiple NICs together for resilience.

Recommendation

• We recommend isolating storage traffic from management traffic. If you are using IP-based storage devices, such as an NFS or iSCSI SR, the storage traffic also flows through the XenServer control domain.
• We recommend that, where appropriate to the storage type, you configure target and host authentication. This helps protect your data if your storage network is exposed to other less trustworthy parts of your organization.Internal attacks on storage networks may enable hostile parties to intercept any data destined for storage. Such data ranges from data that could compromise the integrity of your virtual environment to sensitive data contained within the virtual machines.

NFS

A file-based NFS storage repository mounts the specified NFS repository on the control domain. Each SR is represented as a directory on the remote NFS server, with each virtual disk being a file in that directory stored in the VHD file format.

Recommendation

• We recommend ensuring that, because the VHD is not an encrypted file format, only authorized XenServer hosts and authorized administrators can mount the file system. You can ensure only the correct XenServer hosts can mount the file system by limiting the export to specific IP addresses. Limiting export is typically done when setting up the NFS storage system.
• We recommend ensuring that the storage interfaces of the remote storage device are not visible outside of the dedicated storage network.
• We recommend enabling the NFSv4 AUTH_SYS host authentication.

iSCSI Software Initiator

In the case of iSCSI traffic, XenServer uses the software OpenISCSI initiator to connect to iSCSI targets.

If you are sharing your iSCSI storage network with other XenServer pools or non-XenServer systems, consider separating your iSCSI storage traffic with VLANs.

Recommendation:

• If you have multiple XenServer pools, We recommend that you do not share iSCSI storage networks between pools that have differing levels of trust. For example, a production server pool and a test server pool should not share a storage network.
• We recommend that you configure CHAP target and host authentication unless you are using GFS2.

Fibre Channel

Recommendation:

• We recommend enabling LUN masking on your SAN to secure Fibre Channel traffic.

  LUN masking is a way of configuring a LUN to only accept requests from specific HBAs so that a LUN is available to some hosts and not others. LUN masking shields LUNs from detection when a target is scanned. Depending on your HBA configuration, LUN masking may make it possible to control which hosts can access what volumes on a SAN. LUN masking lets you configure the exact LUNs that a specified host can detect.

• We recommend creating zones in your SAN to separate and secure Fibre Channel traffic.

  SAN zoning is a way to group Fibre Channel devices logically over the storage fabric’s physical configuration. Consider implementing SAN zoning to separate storage traffic from more vulnerable servers (for example, Web servers or other servers with a lot of Internet traffic) from the storage traffic of other servers.

  Zones effectively isolate devices in the zone from devices outside of the zone and prevent devices outside the zone from detecting devices in the zone. Because zoning is implemented in the fabric by the Fibre Channel switch, zoning can control traffic between devices and, in essence, provide a method of access control for SANs. Zones also provide security because zone traffic is isolated from traffic in other zones, which, in turn, limits the amount of traffic compromised in a successful attack.

SMB

If you are sharing your SMB storage network with other XenServer pools or non-XenServer systems, consider separating your SMB storage traffic with VLANs.

Recommendation:

• If you have multiple XenServer pools, We recommend that you do not share SMB storage networks between pools that have differing levels of trust. For example, a production server pool and a test server pool should not share a storage network.
• We recommend that you configure SMB username/password host authentication.