initial discovery failing - is there SDK/plugin needed in vcenter?

[boostedvq35de]

we have 2x Vcenter instances, and although the account works for one instance (vc02) , it's failing for the other(vc01).
I can log into Vcenter UI (on the failing instance, vc01 ) using the credentials.. but netbox-sync is failing:

2023-06-09 22:56:19,688 - INFO: Initializing sources
2023-06-09 22:56:19,688 - DEBUG: Config: source.vc01.enabled = True
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.type = vmware
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.host_fqdn = VC01.XXX.XXX
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.port = 443
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.username = read-only-acct
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.password = read-only-passwd
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.validate_tls_certs = False
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.proxy_host = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.proxy_port = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.permitted_subnets = 172.16.0.0/12, 10.0.0.0/8, 192.168.0.0/16
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.cluster_exclude_filter = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.cluster_include_filter = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.host_exclude_filter = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.host_include_filter = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.vm_exclude_filter = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.vm_include_filter = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.cluster_site_relation = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.host_site_relation = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.cluster_tenant_relation = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.host_tenant_relation = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.vm_tenant_relation = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.vm_platform_relation = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.host_role_relation = .* = Server
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.vm_role_relation = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.cluster_tag_relation = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.host_tag_relation = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.vm_tag_relation = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.match_host_by_serial = True
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.collect_hardware_asset_tag = True
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.dns_name_lookup = True
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.custom_dns_servers = None
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.set_primary_ip = when-undefined
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.skip_vm_comments = False
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.skip_vm_templates = True
2023-06-09 22:56:19,689 - DEBUG: Config: source.vc01.skip_offline_vms = False
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.skip_srm_placeholder_vms = False
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.strip_host_domain_name = False
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.strip_vm_domain_name = False
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.cluster_tag_source = None
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.host_tag_source = None
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.vm_tag_source = None
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.sync_custom_attributes = False
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.host_custom_object_attributes = None
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.vm_custom_object_attributes = None
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.set_source_name_as_cluster_group = False
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.sync_vm_dummy_interfaces = False
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.disable_vlan_sync = False
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.track_vm_host = False
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.overwrite_device_interface_name = True
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.overwrite_vm_interface_name = True
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.host_management_interface_match = management,mgmt
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.ip_tenant_inheritance_order = device, prefix
2023-06-09 22:56:19,690 - DEBUG: Config: source.vc01.sync_vm_interface_mtu = True
2023-06-09 22:56:19,690 - DEBUG: Starting vCenter SDK connection to 'VC01.XXX.XXX'
2023-06-09 22:56:24,765 - ERROR: Unable to connect to vCenter instance 'VC01.XXX.XXX' on port 443. Cannot complete login due to an incorrect user name or password.
2023-06-09 22:56:24,765 - INFO: Source 'vc01' is currently unavailable. Skipping

2023-06-09 22:56:24,765 - DEBUG: Config: source.vc02.enabled = True
2023-06-09 22:56:24,765 - DEBUG: Config: source.vc02.type = vmware
2023-06-09 22:56:24,765 - DEBUG: Config: source.vc02.host_fqdn = VC02.XXX.XXX
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.port = 443
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.username = read-only-acct
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.password = read-only-passwd
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.validate_tls_certs = False
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.proxy_host = None
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.proxy_port = None
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.permitted_subnets = 172.16.0.0/12, 10.0.0.0/8,192.168.0.0/16
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.cluster_exclude_filter = None
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.cluster_include_filter = None
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.host_exclude_filter = None
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.host_include_filter = None
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.vm_exclude_filter = None
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.vm_include_filter = None
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.cluster_site_relation = None
2023-06-09 22:56:24,766 - DEBUG: Config: source.vc02.host_site_relation = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.cluster_tenant_relation = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.host_tenant_relation = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.vm_tenant_relation = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.vm_platform_relation = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.host_role_relation = .* = Server
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.vm_role_relation = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.cluster_tag_relation = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.host_tag_relation = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.vm_tag_relation = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.match_host_by_serial = True
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.collect_hardware_asset_tag = True
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.dns_name_lookup = True
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.custom_dns_servers = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.set_primary_ip = when-undefined
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.skip_vm_comments = False
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.skip_vm_templates = True
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.skip_offline_vms = False
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.skip_srm_placeholder_vms = False
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.strip_host_domain_name = False
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.strip_vm_domain_name = False
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.cluster_tag_source = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.host_tag_source = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.vm_tag_source = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.sync_custom_attributes = False
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.host_custom_object_attributes = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.vm_custom_object_attributes = None
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.set_source_name_as_cluster_group = False
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.sync_vm_dummy_interfaces = False
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.disable_vlan_sync = False
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.track_vm_host = False
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.overwrite_device_interface_name = True
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.overwrite_vm_interface_name = True
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.host_management_interface_match = management,mgmt
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.ip_tenant_inheritance_order = device, prefix
2023-06-09 22:56:24,767 - DEBUG: Config: source.vc02.sync_vm_interface_mtu = True
2023-06-09 22:56:24,768 - DEBUG: Starting vCenter SDK connection to 'VC02.XXX.XXX'
2023-06-09 22:56:26,013 - INFO: Successfully connected to vCenter SDK 'VC02.XXX.XXX'
2023-06-09 22:56:26,013 - INFO: Querying necessary objects from NetBox. This might take a while.

is there any particular plugin/SDK needed ? my ESX admin hasn't been able to identify why the account works from UI access but fails for the script.

[bb-Ricardo]

Hey,

I'm not aware that there are additional components needed.

Which versions of vCenter are you using for vc1 and vc2? Have you tried running it in -l DEBUG3 mode and see how the used credntials look like? maybe there a special characters which need to escpaped?

[boostedvq35de]

thanks -- afaik ESXi 7.0.3 .. I'm not too familiar with the ESX environment.. I did try -l DEBUG3 I only found the following:

send: b'POST /sdk HTTP/1.1\r\nHost: VC01.XXX.XXX\r\nContent-Length: 474\r\nCookie: \r\nSOAPAction: "urn:vim25/7.0.3.0"\r\nContent-Type: text/xml; charset=UTF-8\r\nUser-Agent: pyvmomi Python/3.10.2 (Linux; 4.18.0-408.el8.x86_64; x86_64)\r\nAccept-Encoding: gzip, deflate\r\n\r\n'

send: b'\n<soapenv:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">\nsoapenv:Body<_thistype="SessionManager">SessionManager</_this>
USERNAME
PASSWORD
</soapenv:Body>\n</soapenv:Envelope>'

reply: 'HTTP/1.1 500 Internal Server Error\r\n'

whereas the same works for vc02:

send: b'POST /sdk HTTP/1.1\r\nHost: VC01.XXX.XXX\r\nContent-Length: 474\r\nCookie: \r\nSOAPAction: "urn:vim25/7.0.3.0"\r\nContent-Type: text/xml; charset=UTF-8\r\nUser-Agent: pyvmomi Python/3.10.2 (Linux; 4.18.0-408.el8.x86_64; x86_64)\r\nAccept-Encoding: gzip, deflate\r\n\r\n'

send: b'\n<soapenv:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">\nsoapenv:Body
<_this type="SessionManager">SessionManager</_this>
USERNAME
PASSWORD
</soapenv:Body>\n</soapenv:Envelope>'

reply: 'HTTP/1.1 200 OK\r\n'

I'm trying to push my ESX admin to investigate, but it's taking some time... =(

[bb-Ricardo]

if USERNAME and PASSWORT are the same for both instances then reply: 'HTTP/1.1 500 Internal Server Error\r\n' on the first instance indicates that there is an issue with your vCenter.

[boostedvq35de]

yes, I'm suspecting that is the case.. just how to track down what's causing it... thx