ISAM Desktop SSO, sometimes called Kerberos or SPNEGO authentication, can be difficult to troubleshoot if you make too many assumptions about the source of the problem. I find that the best way to get to resolution is to start at the end and work your way to the beginning. What I mean is that you troubleshoot in the reverse order that you configured ISAM for Desktop SSO.

There are multiple configuration steps required to enable the ISAM WebSEAL server to participate in a Microsoft Active Directory Domain, which enables SSO from the Windows desktop to the ISAM server, providing access to your application.

The diagram below illustrates the process of SPNEGO SSO from a desktop client to ISAM WebSEAL along with steps you can take when troubleshooting the flow.

1. The end user, who is logged in the MSAD domain, makes a request for a protected resource from WebSEAL.  At this point the user is unknown to WebSEAL so they will get an HTTP 401 response when the WebSEAL instance is configured for SPNEGO authentication (which generally implies that Basic Authentication is also enabled).
   Troubleshooting:
   • Verify  the WebSEAL configuration in the [spnego] stanza:
     spnego-auth = https
use-domain-qualified-name = true
spnego-krb-keytab-file = <filename>.keytab
spnego-krb-service-name = HTTP@<hostname>@<MSAD_DOMAIN>
   • If there are multiple server names, the first one MUST match the SPN of the service account.
   • WebSEAL runs on Linux, which is CASE SENSITIVE.  HTTP must be uppercase, the hostname is lowercase and the MSAD Domain is uppercase.
2. The WebSEAL server sends an HTTP 401 response (unauthorized) with two www-authenticate challenge headers:
   www-authenticate: Negotiate
www-authenticate: Basic realm=”IBM Security Access Manager”
   The client browser recognizes the negotiate header because the client browser is configured to support integrated Windows authentication. The client parses the requested URL for the host name.
   Troubleshooting:
   • Verify the browser is configured for Integrated Windows authentication.
   • Verify SPNEGO is enabled for the Local Intranet Zone. If you have multiple domains, ensure the option for “Automatic login with current user name and password” is enabled.
   • If the hostname is in the Trusted Sites Zone, ensure the option “Automatice login with current user name and password” is enabled for this zone as well.
3. If the browser is correctly configured, and the requested hostname is in either the Local Intranet Zone or Trusted Sites Zone, the browser requests a service ticket from the MSAD Domain Controller using the SPN HTTP/<hostname>@<MSAD DOMAIN>. Keep in mind that the SPN is case sensitive as noted in step 1 and the MSAD Domain component will be that of the user.
   Troubleshooting:
   • Verify the functional user created in MSAD for SPNEGO matches the SPN:
     setspn -L <MSAD Domain>\<functional account>
4. The MSAD Domain Controller returns a service ticket to the client browser.
5. The client browser makes the request again, supplying the service ticket in the authorization header.
6. WebSEAL contacts the MSAD Domain Controller to validate the service ticket using the keytab file to authenticate and decrypt the service ticket.  As part of this validation DNS is used to verify the hostname can be resolved in DNS both forward and reverse .
   Troubleshooting:
   • Using the LMI:
     Navigate to Secure Web Settings > Global Settings > Kerberos Configuration
     Click Test on any of the tabs other than Keyfiles. This test checks whether the Kerberos principal for the WebSEAL instance can authenticate to the KDC.
   • Click Test on the Keyfiles tab to verify that the WebSEAL instance can use the service name to authenticate to the KDC.
   • Verify with NSLookup that the hostname resolves to an IP address.
   • Verify with NSLookup that the returned IP address resolves back to the hostname.
7. The MSAD Domain Control returns the user information. The end result is an authenticated user session in WebSEAL
8. WebSEAL verifies the user’s authorization to access the protected resource and then requests the protected resource from the application server, supplying user information in any defined junction parameters or object definitions.
9. Application server returns the protected resource to WebSEAL.
10. WebSEAL returns the protected resource to the browser along with the WebSEAL session cookie.

All future requests to the WebSEAL server will include the WebSEAL session cookie so the SPNEGO challenge does not need to be repeated.

One of the problems with troubleshooting this flow is that SPNEGO challenge and response is not visible to the client browser because it is handled by the Windows operating system. This means that you will not see the initial challenge in Step 2 or the re-submitted request in step 5 even when you enable developer mode in the client browser.

If Step 3 or Step 6 fail, you will see either the login page from WebSEAL or a Basic Authentication challenge. Follow the troubleshooting and verification procedure for each step until you determine the problem.

When the user is not authorized to access the protected resource you will get an HTTP 403 (Unauthorized) response or the configured error page.

The downstream server can also return a response other than one in the 200 range. This will also generate an error to the user unless the response-code-rules have been modified. Keep this in mind when building a REST application that expects to get the actual response code from the juntioned server.

In order for the browser to be able to provide valid Kerberos credentials to ISAM, both the browser and ISAM WebSEAL must be in the same MSAD Domain. This is one of the common problems I see in organizations that have multiple MSAD domains: The tester’s workstation is not in the same MSAD domain as the ISAM WebSEAL instance is configured for.

Resources

• SPNEGO protocol and Kerberos authentication
• Configure Windows desktop single sign-on
• Single sign-on Issues: Windows Desktop single sign-on, Kerberos, and SPNEGO