Veeam Backup & Replication | Immutable Backup

  • Post author:
  • Reading time:20 mins read
  • Post category:Veeam
  • Post comments:0 Comments

Hardened Linux Repository

Hello there everyone.

Today, I want to check out the “Hardened Linux Repository” feature that was introduced in V11 (I think) for immutable backups. Immutable backups was already a thing before, but it utilized S3 and other object storage providers.

Don’t know if “minio” would work. Could be another project, I guess.

So, the plan is to deploy a Linux OS, RHEL in my case, and see how to set up the immutable backup.

Linux Installation and Configuration

I will not go through the installation process, did this a few times already. So I assume that the system is already running.

Here is what I deployed.

  • OS: RHEL 9 – (Minimal Install)
  • RAM: 8GB
  • CPU: 2 cores
  • Disk1: sda 30GB – root
  • Disk2: sdb 50GB – Backup repository

Alright. Let’s start with the configuration.

First, I will create a user that will be used by veeam for the backups. I will also give it sudo permissions, which will be removed later.

RHEL :: ~ » sudo useradd veeam 
RHEL :: ~ » sudo passwd veeam
Changing password for user veeam.
New password: 
Retype new password: 
passwd: all authentication tokens updated successfully.
RHEL :: ~ » sudo usermod -a -G wheel veeam

Now create the filesystem on the second disk (sdb). Make sure you select the correct one. We can check this, with for instance “lsblk”.

These are the recommended settings for the filesystem. Those should be the default though. So a basic “mkfs.xfs /dev/sdb” should be fine. But, just to make sure, I will type it out.

RHEL :: ~ » sudo mkfs.xfs -b size=4096 -m reflink=1,crc=1 /dev/sdb -f
meta-data=/dev/sdb               isize=512    agcount=4, agsize=3276800 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1    bigtime=1 inobtcount=1
data     =                       bsize=4096   blocks=13107200, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=6400, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Next, I will create a folder for the backups.

RHEL :: ~ » sudo mkdir /mnt/VEEAMREPO

We will need to mount our disk to this new location.

RHEL :: ~ » sudo mount /dev/sdb /mnt/VEEAMREPO

Give the correct permissions.

RHEL :: ~ » sudo chown -R veeam.veeam /mnt/VEEAMREPO
RHEL :: ~ » sudo chmod 700 /mnt/VEEAMREPO

Also, make sure to add the mount point to the fstab file. Otherwise, it won’t mount the disk after a reboot.

You can get the “UUID” with the “blkid” command.

RHEL :: ~ » sudo blkid
/dev/sdb: UUID="e28cf5ab-8ae5-4ef9-a7fe-5a1ee946a6ca" TYPE="xfs"
RHEL :: ~ » sudo vim /etc/fstab
UUID="e28cf5ab-8ae5-4ef9-a7fe-5a1ee946a6ca" /mnt/VEEAMREPO xfs defaults 0 0

That should be it for the basic setup.

Veeam Backup & Repository Configuration

Adding Linux Server to Veeam Backup & Replication

Let’s add the Linux server to Veeam.

Navigate to “Backup Infrastructure” and select “Managed Servers”. Right-click and select “Add Server”.

Select “Linux”.

Enter the IP or hostname of the Linux server.

Create a “Single-use credentials for hardened repository” entry by clicking on “Add..”. We need a user with elevated permissions because Veeam needs to install the transport services first.

Apply the settings and click on “Finish” once the installation is done.

Add a new Backup Repository

Now we can add the new system as a backup target.

Navigate to “Backup Repositories”, right-click and select “Add backup repository”.

Select “Direct attached storage” -> “Linux”.

Give it a fitting name.

Select the “Repository server” we just added, there is only one in my case, and click on “Populate”.

Choose the mount point “/mnt/VEEAMREPO” and click on “Next”.

Select “Use fast cloning on XFS volumes”, “Make recent backups immutable for X days” and click on “Next”.

I will leave mount server for restores at the defaults.

Then “Apply” -> “Next” and “Finish”.

OK. We have our repository now. Let’s test it.

Creating a Backup Job

Let’s remove the sudo permissions for the “veeam” user first.

RHEL :: ~ » sudo gpasswd -d veeam wheel

Now, navigate to “Home” and click on “Backup Job” -> “Virtual machine…”.

Choose a name.

Click on “Add” to select a virtual machine. I will select only one for this demonstration.

Select your “backup repository” and the “Retention policy”. The retention has to be higher than the immutable period.

I will enable “application-aware processing”, though in this case, it wouldn’t really be relevant.

In the last step, we define the schedule and click on “Apply”.

Testing the Backup and Immutability

That’s it. Let’s start the job.

Worked flawlessly. Next, I want to check if the files actually are immutable. First, I want to check on the Linux server itself.

To see if the flag actually is set, we can use the “lsattr” command.

RHEL :: ~ » cd /mnt/VEEAMREPO/backups/Immutable-backup-job
RHEL :: Immutable-backup-job » lsattr 
----i----------------- ./Immutable-backup-jobD2022-12-29T121401_A584.vbk
----i----------------- ./Immutable-backup-jobD2022-12-29T121923_CBE9.vib
---------------------- ./Immutable-backup-job.vbm

We can see that the “i” flag is set, but only for the vbk and vib files. Since the vbm changes on every backup, it’s not possible to set the flag on this file. This file is required for performing file-level and volume-level restore operations, but in the worst case, you should be able to import the vbk file and restore the whole VM.

The “i” flag.

Files with this attribute cannot be deleted or modified, and you cannot create a hard link to the file. Symbolic links (soft links) are still possible. This attribute can be set and removed only with root privileges.

Next, testing it in Veeam.

Great. It won’t not allow the deletion, until the set amount of time expires (14 days in my case).

Hardening recommendations

There are a few more recommendations by Veeam for hardening the Linux system.

First, change the file permissions for the authentication certificate. This allows only the “Veeam Data Mover” to read that file.

# Create the folder, if it does not exist already
RHEL :: ~ » sudo mkdir -p /opt/veeam/transport/certs
# Change the owner
RHEL :: ~ » sudo chown veeam.veeam /opt/veeam/transport/certs
# Set the permissions for that folder
RHEL :: ~ » sudo chmod 700 /opt/veeam/transport/certs

Disable SSH on the repository server. Since it’s only required for deployment and upgrade of the Veeam Data Mover.

You probably should only do this, if you have physical access.

# Prevent start on boot
RHEL :: ~ » sudo systemctl disable sshd
# Stops the service
RHEL :: ~ » sudo systemctl stop sshd

Alright. That’s it for the immutable backup repository. Hope this helps.

Till next time.

Leave a Reply