"cacheHostInfo is null" when removing a server from a SharePoint 2013 farm

I kept getting the error "cacheHostInfo is null" when trying to remove a web front-end server from a SharePoint 2013 farm. The error came up in both Central Administration from the Servers in Farm page and PowerShell from the SPServer.Delete() method.

The issue was that the Distributed Cache instance on the WFE was still registered. Raka Satria wrote about a similar issue when trying to remove the Distributed Cache instance from a server in SharePoint 2013 SPDistributedCacheServiceInstance "cacheHostInfo is null"

I borrowed the PowerShell he provides to resolve his issue and modified it for my scenario:

$SPFarm = Get-SPFarm
$cacheClusterName = "SPDistributedCacheCluster_" + $SPFarm.Id.ToString()
$cacheClusterManager = [Microsoft.SharePoint.DistributedCaching.Utilities.SPDistributedCacheClusterInfoManager]::Local
$cacheClusterInfo = $cacheClusterManager.GetSPDistributedCacheClusterInfo($cacheClusterName);
$instanceName ="SPDistributedCacheService Name=AppFabricCachingService"
$serviceInstance = Get-SPServiceInstance | ? {($_.Service.Tostring()) -eq $instanceName -and ($_.Server.Name) -eq "Server name for server you are trying to remove"}
$serviceInstance.Delete()

Once the Distributed Cache instance was deleted I could remove the server from the farm.