A significant challenge in disaster scenarios is that backup solutions like Veeam often lack redundancy. This leads to situations where hardware failures or corrupt updates can incapacitate backup systems, leading to the impossibility to perform recovery operations within a certain timeframe.
To mitigate this, along with patch management and appropriate hardware care packages, the architecture should be adjusted to meet state-of-the-art requirements. This involves examining and defining Veeam’s own Recovery Time Objective (RTO) to minimize the RTO for entire infrastructures in turn.
For customers I implement the following variations based on their preferences and possibilities:
Option 1: Config Backup with a prepared VBR server
Option 2: Veeam Replication (preferred)
Option 1:
If a client, for instance, cannot or does not wish to virtualize the Veeam backup server, the Veeam instance, including its database, will be installed on a physical machine. This means losing access to Veeam as soon as the relevant hardware fails or is defective. Consequently, you cannot immediately continue working with additional backup repositories (NAS, Cloud, etc.) because the Veeam instance itself is missing.
To avoid this and keep the RTO as low as possible, you could prepare a Veeam instance on another physical or virtual machine. This „standby system“ is fully installed but not yet configured. All functions are set up, Veeam including its underlying database is installed, and network configurations and hardening are performed so that in the event of a failure, you only need to apply the Veeam Configuration Backup and can then immediately continue with the new instance.
It’s crucial to ensure access to the Veeam Configuration Backup, so it should be stored on an external repository and not on the server hosting the Veeam instance.
Note:
With this option, it’s important to remember that a prepared „standby system“ cannot have the same IP address and hostname as the production system. This means that in the event of a failure, the IP address must be changed to that of the production system, which could damage the Veeam instance or require additional configurations.
If the system has a different IP address and hostname, appropriate firewall rules must be prepared so that this system can operate fully after the Configuration Backup is applied. Additional configurations within Veeam might also be necessary.
Option 2:
Veeam offers VM replication capabilities. For most customers, I virtualize the Veeam installation (Veeam Backup server), allowing Veeam to benefit from cluster advantages. This includes load balancing, virtual resource adjustment, and using replication to minimize RTO.
Important: Although this approach has been tested, it is not an official procedure from Veeam!
The procedure involves replicating the VM hosting the Veeam installation to local datastores of hypervisor servers in different data centers using Veeam Replication. Depending on the environment and the number of backup jobs, and depending on the setup, an interval of 15-60 minutes is recommended, with retention not needing to be particularly long.
Thus, if the storage hosting the Veeam VM fails, you can start the Veeam VM directly on one of the hypervisor servers according to a predetermined emergency plan, as it exists there as a replica, as a startable VM.
Veeam typically requires that a replica be started via the Veeam console, which, however, is not possible in this case.
The following is an example of what I have configured in a vSphere environment:
I configured 30 minutes as replication interval:
The completed replication can also be monitored in Veeam:
Important: The replica must not be started from here, as the service required for this is also controlled by Veeam. However, the VM – and thus the service – will be shut down as part of the failover process.
On the host, the entire setup looks as follows:
The operational, replicated, and ready-to-start VM resides on the local ESXi host and can also be launched from there should the vCenter fail:
Note: Adding ESXi hosts in Veeam:
In the event of a disaster, it is expected that both the vCenter and Domain Controllers might fail, leaving central services like DNS and the connection from Veeam to the virtualization environment unavailable.
Typically – and still the best approach in most cases – the environment is connected to Veeam via vCenter. This means, however, that Veeam looks to restore the connection through the vCenter. If this is unavailable, the restore will encounter errors.
To avoid this, I also add each host as a standalone host via its IP address, in addition to the vCenter connection, to enable restoration to a dedicated host independently of DNS in an emergency. For the scenario mentioned above, I also use the element created by IP address as the target host. In the Veeam console, this looks as follows:
I also apply this in Hyper-V environments!
Tests conducted in collaboration with various customers – in environments of varying sizes – have shown that no obvious errors occur that would impair the functionality of Veeam when operating the replica VM.
Important: A Configuration Backup in Veeam is absolutely essential in any case!