What a mouthful! So I was changing roles and deployment on our Cisco ISE back end yesterday (2.3.0.298) when I ran into an interesting visual error on the primary administration node:

ise

Turns out in your DNS server the reverse lookup has to match the hostname (A Records) of the ISE node(s). So my issue was that i had two zones and the ISE names and the elasticsearch component of ISE was picking up the non primary zone during a reverse DNS lookup. I removed those records and the error disappears and the correct data shows in the work center view.

The command I used to troubleshoot from the primary administration node shell was:

show application logging ise-elasticsearch.log

The dead give away was in the log file:

[2018-11-14 01:00:12,976][INFO ][discovery.zen ] [ibra] failed to send join request to master [{ithaca}{ROGHmWtxSv6JWzhjW3maeQ}{10.160.119.174}{ithaca.domain.com/10.160.119.174:9300}], reason [NodeDisconnectedException[[
ithaca][ithaca.domain.com/10.160.119.174:9300][internal:discovery/zen/join] disconnected]] [2018-11-14 01:00:16,077][WARN ][plugin.ssl.transport ] [ibra] exception caught on transport layer [[id: 0x7ce0f56d, /10.160.15.174:23637 => ithaca.domain.com/10.160.119.174:9300]], closing connection