This here says you can go into regedit to remove the interfaces, but this doesn't appear to correct the issue. (new switches being created at every reboot)
The virtual switch configuration stored within the following key: HKLM\SYSTEM\CurrentControlSet\Services\VMSMP\Parameters\SwitchList After they deleted the corresponding value within that key and rebooted, the virtual switch was gone. Source: Can't remove failed virtual switch from Hyper-V's Virtual Switch Manager https://social.technet.microsoft.com/Forums/windows/en-US/e49df568-4f4c-47b7-b30c-952d1e26ca58/cant-remove-failed-virtual-switch-from-hypervs-virtual-switch-manager