My experience of vulnerability disclosure

Mat Mannion -

This story describes my experience in researching and attempting to disclose the vulnerability described as JolokiaPwn.

I've been recently taking a few information security courses to try and shore up my self-taught knowledge with actual qualifications. I wanted to go down an industry-standard route so I've been taking CREST-accredited courses from 7Safe, which have been informative and enjoyable even if it does, to an extent, cover existing ground.

While taking one of these in late May, I was reminded about a vulnerability I'd found in our own architecture at Warwick last November when I was playing with trying to expose some JMX statistics on our JDBC data sources over Jolokia. To cut a long story short, after working out I could trivially construct a URL that would (through Jolokia) dump database URLs and credentials in plain text as JSON, I realised that during an upgrade of Jolokia we'd lost the mandatory customisation of the WAR agent to enable authentication, and weren't blocking requests at our load balancer, meaning that it was possible to create a predictable URL to dump information from all of our apps.

via GIPHY

Remediation wasn't difficult; re-package the Jolokia WAR agent with edits to web.xml based on the instructions in the Jolokia manual, block requests to /jolokia that aren't from localhost (for our monitoring), check access logs since we'd introducted the vulnerability to make sure there'd been no requests, and then forget about it.

For six months I didn't really think about it, until something reminded me on the training course I was on. I spent a couple of hours writing a programme that would download the Alexa top million domains, and fire a request at each of them with /jolokia on the end to see if the Jolokia agent responded with information about the running version and application server. I left it running for a few days, and came back to find that I wasn't the only person who'd made such a mistake. In fact, I'd found about 150 domains from all across the spectrum, from tourism sites all the way up to major financial services sites and even banks across the world.

The trainer on the course I was on had, coincidentally, talked about disclosing vulnerabilities through HackerOne and after a little bit of time trying to work out how that worked, and reaching out to Scott Helme on Twitter for some advice on how I should go about the process of disclosure and who to, I submitted a request for disclosure assistance. I was pretty sure at this point that I'd need help contacting the 150 or so companies that I'd found the serious vulnerability with.

Having slept on it, I changed my mind, and started the process of finding contacts for each domain, either a technical contact on their site (and some information about a security disclosure policy). Naively, I expected most sites to publish this information somewhere accessible, but no such luck. I maybe found security contact details on websites for 1 or 2 sites at most, the majority I resorted to contacting either via a contact form on their website or via Twitter DM. I spent pretty much my entire weekend doing this, with the fanciful idea that it would be treated as a major incident by these companies' out of hours support staff and I'd be working through resolution, but I only ended up in touch with an appropriate information security professional at one company.

For every company I contacted, I was asking two simple questions.

  1. Could you give me contact details for your security incident response team?
  2. Do you have a disclosure policy and/or subscribe to a bug bounty program?

Of those that replied, most put me in touch with someone who wouldn't deal with me any further until I'd given them more information. To these people I disclosed the bug in full (I don't think there's any real merit in partial disclosure, and it would have been all but impossible in this situation). One organisation (through a white-label affiliate) had a private program on HackerOne that they were able to invite me to and pretty swiftly verified and resolved the issue, paying an appropriate bug bounty. Unfortunately, this organisation was very much the exception; even major organisations such as banks or major financial services companies either refused to pay bug bounties, or promised details of ad-hoc bug bountry programs before going completely silent and not answering emails any more. Frustrating, but it's not like I'm in it for the money anyway, I was just hoping to be compensated for the time (and stress) of disclosing the vulnerability responsibly.

Going back towards the start of this, I contacted both the Jolokia maintainer and the Apache Tomcat security team as the first two on my list. I was pretty sure that neither would consider this to be a problem in their software, just that they were the messenger - indeed, Tomcat's own documentation describes JMX access as equivalent to root on the box:

Tomcat exposes a large amount of internal information and control via JMX to aid debugging, monitoring and management. Given the limited access control available, JMX access should be treated as equivalent to local root/admin access and restricted accordingly.

As expected, Tomcat simply stated they do not consider it a vulnerability. The Jolokia maintainer was able to get back to me after a week or so, and said something similar; he was disappointed to hear of the scale of the problem of installs of Jolokia without proper access control, but Jolokia is acting very much as intended.

I suppose this disclosure is slightly strange in that it doesn't represent a bug in software per se, but instead misconfiguration - it's much more akin to default accounts being left enabled. The impact just happens to be vast in this example. Perhaps I've been overly naive in the way I've approached the disclosure, but I'm left more than a little turned off by the way most commercial organisations handled my report; many simply closed the vulnerability once they were given the information without even acknowledging that it happened, while some still haven't responded at all. If I were to give my opinion on how these companies could improve, I would say:

  • Provide security contacts to your website somewhere searchable, or at least available to your support staff. The proposal for security.txt could represent a common mechanism to publish this
  • Subscribe to a vulnerability disclosure program on HackerOne or similar to manage reports. You don't need to offer bug bounties if you don't agree with them, but what's the harm in providing a mechanism to receive reports?