Using the UK federation Test IdP in the list of IdPs to probe but removing that IdP's metadata from the SP leads to
2019-06-25 07:38:13 WARN Shibboleth.SessionInitiator.SAML2 [4] [default]: unable to locate metadata for provider (https://test-idp.ukfederation.org.uk/idp/shibboleth)
Probe doesn't recognise this is a problem with the SP and reports lines like
2019-06-25T07:35:44Z, https://test-idp.ukfederation.org.uk/idp/shibboleth, 500
which are indistinguishable from results where the metadata hasn't propagated to the IdP yet.