Omnissa DEM – Ubuntu File Server Setup

As part of setting up Omnissa Dynamic Environment Manager (DEM) in my lab environment, I needed a file server to host the DEM configuration share, profile archive share, and folder redirection share. Previously I have used a Windows Server for this, posted here: Omnissa DEM – Prepare File Shares and Permissions.

This time, rather than using a Windows file server, I decided to go the Linux route and set up a domain-joined Ubuntu 24.04 LTS server running Samba. This turned out to work very well, and the shares can be managed from Windows just like any other file server. This page documents the full setup from scratch.

The diagram below shows how the shares, subfolders, and AD user groups relate to each other. Dashed lines show which groups have access to which folders. The dem-admins group has full control everywhere. Each pool’s entitlement group gets read access to its own config subfolder and read/write access to its own profile subfolder. Domain Users can only create their own subfolder in the hzredir$ root — nothing more.

Environment

Shares

Three hidden SMB shares are created on the file server. Each share serves a different purpose in the DEM setup. The demcfg$ and demprf$ shares are organised with a subfolder per desktop pool, so different pools get completely isolated DEM environments. The hzredir$ share uses a flat structure where Windows/DEM automatically creates a per-user subfolder on first login using the %USERNAME% variable.

Prerequisites

• Static IP assigned to the server
• DNS A record and PTR record created before starting
• Domain Administrator credentials available
• Second virtual disk added for DEM data (optional but recommended)

Step 1 – OS Installation

Install Ubuntu 24.04 LTS Server with the default guided storage layout (LVM, ext4) and set the hostname during the install wizard. Once the installation is complete, the first thing to fix is /etc/hosts. The Ubuntu installer places a loopback entry for the hostname using 127.0.1.1, which looks like this:

127.0.1.1    fileserver

This needs to be commented out or removed and replaced with the server’s actual static IP address, FQDN, and short hostname. This is important because Samba and Kerberos both do hostname lookups during domain join and authentication — if the server resolves its own name to a loopback address instead of its real IP, the domain join will either fail or produce hard-to-diagnose authentication errors later.

Open the file with:

sudo nano /etc/hosts

Comment out the 127.0.1.1 line and add the correct entry below it:

# 127.0.1.1    fileserver
x.x.x.x    fileserver.domain.local    fileserver

Verify with:

hostname -f

This should now return the full FQDN, e.g. fileserver.domain.local. If it still returns just the short hostname, the /etc/hosts entry is not correct.

Step 2 – Data Disk Setup

Format the second disk as XFS and mount it at /srv/dem. XFS is well suited for this workload because DEM profile archives are ZIP files accessed concurrently by many user sessions at the same time. XFS uses independent allocation groups that can be addressed in parallel by the kernel, meaning multiple concurrent read/write operations do not contend with each other. This is covered in the Red Hat Enterprise Linux filesystem documentation, which notes that XFS performs well on systems with multi-threaded, parallel I/O workloads.

sudo mkfs.xfs /dev/sdb
sudo mkdir -p /srv/dem
echo "/dev/sdb /srv/dem xfs defaults,noatime,uquota 0 2" | sudo tee -a /etc/fstab
sudo systemctl daemon-reload
sudo mount -a

Step 3 – Install Samba and Winbind

sudo apt update && sudo apt upgrade -y
sudo apt install -y samba winbind libpam-winbind libnss-winbind krb5-user

When prompted for the Kerberos realm, enter your domain in uppercase, e.g. DOMAIN.LOCAL.

Step 4 – Configure Kerberos

sudo tee /etc/krb5.conf << 'EOF'
[libdefaults]
    default_realm = DOMAIN.LOCAL
    dns_lookup_realm = false
    dns_lookup_kdc = true
[realms]
    DOMAIN.LOCAL = {
        kdc = dc01.domain.local
        admin_server = dc01.domain.local
    }
[domain_realm]
    .domain.local = DOMAIN.LOCAL
    domain.local = DOMAIN.LOCAL
EOF
kinit Administrator@DOMAIN.LOCAL
klist

Step 5 – Configure Samba and Join Domain

Configure /etc/samba/smb.conf with security = ADS, winbind RID mapping, and the three DEM shares. Start smbd and nmbd first, then join the domain and start winbind. Important: add the cifs SPN after joining and recreate the keytab — without this, Kerberos authentication from Windows clients will fail.

sudo tee /etc/samba/smb.conf << 'EOF'
[global]
    workgroup = DOMAIN
    realm = DOMAIN.LOCAL
    security = ADS
    kerberos method = secrets and keytab
    idmap config * : backend = tdb
    idmap config * : range = 10000-19999
    idmap config DOMAIN : backend = rid
    idmap config DOMAIN : range = 20000-99999
    winbind use default domain = yes
    winbind enum users = yes
    winbind enum groups = yes
    winbind refresh tickets = yes
    vfs objects = acl_xattr
    store dos attributes = yes
    server min protocol = SMB2
    server signing = auto
    log file = /var/log/samba/log.%m
    log level = 1
[demcfg$]
    path = /srv/dem/demcfg
    read only = yes
    browseable = no
    valid users = "@DOMAIN\dem-admins" "@DOMAIN\pool1-users" "@DOMAIN\pool2-users"
    write list = "@DOMAIN\dem-admins"
    create mask = 0664
    directory mask = 0775
[demprf$]
    path = /srv/dem/demprf
    read only = no
    browseable = no
    valid users = "@DOMAIN\dem-admins" "@DOMAIN\pool1-users" "@DOMAIN\pool2-users"
    write list = "@DOMAIN\dem-admins" "@DOMAIN\pool1-users" "@DOMAIN\pool2-users"
    create mask = 0600
    directory mask = 0700
    hide unreadable = yes
[hzredir$]
    path = /srv/dem/hzredir
    read only = no
    browseable = no
    valid users = "@DOMAIN\dem-admins" "@DOMAIN\Domain Users"
    write list = "@DOMAIN\dem-admins" "@DOMAIN\Domain Users"
    create mask = 0600
    directory mask = 0700
    hide unreadable = yes
EOF
sudo systemctl restart smbd nmbd
sudo net ads join -U Administrator
sudo net ads setspn add cifs/fileserver.domain.local -U Administrator
sudo net ads setspn add cifs/FILESERVER -U Administrator
sudo net ads keytab create
sudo systemctl start winbind

Verify AD integration:

wbinfo -p
wbinfo -u
wbinfo -g

Step 6 – Create Folder Structure

Create the share root directories and per-pool subfolders. Set ownership to administrator/domain admins so Windows ACLs can be applied. Set 0771 on the share roots so domain users can traverse into their pool subfolders without being able to list the root.

sudo mkdir -p /srv/dem/demcfg /srv/dem/demprf /srv/dem/hzredir
sudo chown "administrator":"domain admins" /srv/dem/{demcfg,demprf,hzredir}
sudo chmod 0771 /srv/dem/{demcfg,demprf,hzredir}
sudo mkdir -p /srv/dem/demcfg/{pool1,pool2}
sudo mkdir -p /srv/dem/demprf/{pool1,pool2}
sudo chown "administrator":"domain admins" /srv/dem/demcfg/{pool1,pool2}
sudo chown "administrator":"domain admins" /srv/dem/demprf/{pool1,pool2}

Step 7 – Set NTFS ACLs from Windows

Run the following PowerShell script as Domain Admin to apply the correct Omnissa-recommended ACLs on each pool subfolder. The script is parameterized so new pools can be added easily later.

[CmdletBinding()]
param (
    [string]$Server = "fileserver.domain.local",
    [string]$ConfigShare = "demcfg$",
    [string]$ProfileShare = "demprf$",
    [string]$DemAdminsGroup = "DOMAIN\dem-admins",
    [array]$Pools = @(
        @{ Name = "pool1"; EntitlementGroup = "DOMAIN\pool1-users" },
        @{ Name = "pool2"; EntitlementGroup = "DOMAIN\pool2-users" }
    )
)
$configRoot  = "\\$Server\$ConfigShare"
$profileRoot = "\\$Server\$ProfileShare"
foreach ($pool in $Pools) {
    $poolName = $pool.Name
    $entGroup = $pool.EntitlementGroup
    # Config folder - read only for entitlement group
    $cfgPath = "$configRoot\$poolName"
    New-Item -ItemType Directory -Path $cfgPath -Force | Out-Null
    $cfgAcl = Get-Acl $cfgPath
    $cfgAcl.SetAccessRuleProtection($true, $false)
    $cfgAcl.AddAccessRule((New-Object System.Security.AccessControl.FileSystemAccessRule("SYSTEM","FullControl","ContainerInherit,ObjectInherit","None","Allow")))
    $cfgAcl.AddAccessRule((New-Object System.Security.AccessControl.FileSystemAccessRule($DemAdminsGroup,"FullControl","ContainerInherit,ObjectInherit","None","Allow")))
    $cfgAcl.AddAccessRule((New-Object System.Security.AccessControl.FileSystemAccessRule($entGroup,"ReadAndExecute","ContainerInherit,ObjectInherit","None","Allow")))
    Set-Acl -Path $cfgPath -AclObject $cfgAcl
    Write-Host "Config folder OK: $cfgPath" -ForegroundColor Green
    # Profile folder - CREATOR OWNER gets full control on subfolders only
    $prfPath = "$profileRoot\$poolName"
    New-Item -ItemType Directory -Path $prfPath -Force | Out-Null
    $prfAcl = Get-Acl $prfPath
    $prfAcl.SetAccessRuleProtection($true, $false)
    $prfAcl.AddAccessRule((New-Object System.Security.AccessControl.FileSystemAccessRule("SYSTEM","FullControl","ContainerInherit,ObjectInherit","None","Allow")))
    $prfAcl.AddAccessRule((New-Object System.Security.AccessControl.FileSystemAccessRule($DemAdminsGroup,"FullControl","ContainerInherit,ObjectInherit","None","Allow")))
    $prfAcl.AddAccessRule((New-Object System.Security.AccessControl.FileSystemAccessRule("CREATOR OWNER","FullControl","ContainerInherit,ObjectInherit","InheritOnly","Allow")))
    $prfAcl.AddAccessRule((New-Object System.Security.AccessControl.FileSystemAccessRule($entGroup,"ReadAndExecute, CreateDirectories","None","None","Allow")))
    Set-Acl -Path $prfPath -AclObject $prfAcl
    Write-Host "Profile folder OK: $prfPath" -ForegroundColor Green
}
Write-Host "Done." -ForegroundColor Cyan

For the hzredir$ share root, use icacls since Set-Acl can struggle with CREATOR OWNER over SMB:

$share = "\\fileserver.domain.local\hzredir$"
takeown /F $share /A
icacls $share /inheritance:d
icacls $share /reset
icacls $share /grant "SYSTEM:(OI)(CI)F"
icacls $share /grant "DOMAIN\dem-admins:(OI)(CI)F"
icacls $share /grant '"CREATOR OWNER":(OI)(CI)(IO)F'
icacls $share /grant "DOMAIN\Domain Users:(NP)(RX,WD)"

GPO Configuration

For each desktop pool, configure the following settings under User Configuration > Policies > Administrative Templates > Omnissa DEM > FlexEngine. The config and profile paths include a per-pool subfolder so different pools get isolated DEM environments.

Adding a New Pool

To add a new desktop pool later, three things need to be done. First, create and set ownership on the new subfolders from Linux. Second, add the new entitlement group to valid users and write list in /etc/samba/smb.conf and reload Samba. Third, run the PowerShell script with the new pool as a parameter:

.\Set-DEMShares.ps1 -Pools @(
    @{ Name = "newpool"; EntitlementGroup = "DOMAIN\newpool-users" }
)

Disclaimer: Every tips/tricks/posting I have published here, is tried and tested in different IT-solutions. It is not guaranteed to work everywhere, but is meant as a tip for other users out there. Remember, Google is your friend and don’t be afraid to steal with pride! Feel free to comment below as needed.