splunk> authentication fall down go boom

So I went to log onto my splunk> app the other day, only to be slapped in the face with "Failed to set cookie. Ensure cookies are enabled in your browser" when I tried to authenticate. Having only just logged on a couple of hours before, using the same workstation and browser, I was a little boggled by this. I switched over to IE, tried to log on again, and got the same error. Hmmm. So then I tried to logon with my local admin account. Uhm, yeah, about that. When you set a password for that user, you kinda ought to make sure you can remember it. D’oh!

Long story short, closed, opened, cleared cache, verified cookies accepted from the intranet zone, added to trusted sites, rebooted, and still I got the same error! Then I tried to logon from another workstation and hit the same brick wall. So I did a search on Google, and a search of Splunk’s knowledgebase; both to no avail. Which of course, once I finally did figure out the problem, prompted me to write this post.

Despite what the error text implies, this has NOTHING to do with your browser and cookies. For reasons I cannot explain, splunk> throws this error when it cannot authenticate against an external source. If you configured splunk> to authenticate against Active Directory then you set it to use LDAP over SSL. This of course requires the domain controllers to actually support LDAP over SSL, and that is where things went terribly wrong for me. disclaimer….I didn’t set up the Active Directory, or the Certificate Authority for this domain. Sorry, but I do have a reputation to maintain, tarnished though it is by the fact that I should have noticed this before now.

I configured splunk> to use the domain FQDN instead of a specific host, thinking that would provide fault tolerance, and a minimal latency since we have fast WAN links and only a couple of locations. As things turned out, only one domain controller was actually set up for LDAPS, its certificate expired, and the CA was not configured to automatically renew/issue domain controller certificates. So when the certificate on the one DC expired, splunk> would not connect using LDAPS with an invalid cert. No other domain controller was actually running LDAPS, so AD authentication was toast. Why this generated the error "Failed to set cookie. Ensure cookies are enabled in your browser" instead of something like "authentication method failed" or other more accurate wording remains a mystery.

So I updated the domain controller certificate template, added the domain controllers to the Certificate Service DCOM Access group, and installed new certs on each DC. Problem solved. I was then able to log on to splunk>. And yes, I then pointed splunk> to a specific DC in the same site, and reset the local account password just in case this happens again!

And this wouldn’t be a RetoHack post if I didn’t close with some words of wisdom. Since we started down this path speaking of cookies, always remember, people should not be afraid of cookie, cookie should be afraid of people.

Direct link for RSS and email subscribers…http://www.youtube.com/watch?v=V9rzMaAucI4

Too obscure? Think Hugo Weaving’s second best movie ever…"V For Vendetta." Of course, this was his best. Sure, he rocked as Elrond, and Agent Smith was a badass, but he will always be Tick to me.

You might also enjoy:

1. Getting started with Splunk…using Active Directory authentication
2. howto://Configure Splunk to use AD groups
3. Enabling Filezilla Server integration with Active Directory authentication
4. howto://Install Splunk

Tagged as: active directory, splunk>