New Beginnings

So, after 7 intense years at Infratects (now Inlumi), it's time for me to move on.

I have a few ideas about what to do next, but nothing set in stone yet. My LinkedIn profile could do with more details, but it's a decent primer for what I do for a living - Hyperion/EPM, Python, Java, Weblogic and thereabout.

I was a web developer in a previous life, so I enjoy hacking and automating everything, getting dirty with infrastructure and the cloud; and in 13 years working on Hyperion products, I've absorbed a pretty good amount of knowledge related to financial processes (consolidation rules, metadata, cube performance and so on) as well as a deep understanding of the innards of the EPM suite. I can tweak your database, hack your Weblogic, integrate your cloud-based authentication, script your exports and migrations, and so on; and if there is something the tools won't do... I'll build you a new tool! That's where I make the difference: at the intersection of technology and Finance, boosting the productivity of accountants.

If this sounds interesting, ping me on LinkedIn and we can have a chat.

Link feeds about Weblogic, EPM, Hyperion etc

I've recently started using the excellent Pinboard bookmark manager (a wonderful throwback to the glory days of del.icio.us) to keep track of interesting posts. While I think about the best way to syndicate those links across all my public accounts, you can check out these feeds:

• Everything -- https://pinboard.in/u:toyg/ [ RSS ]
• Python -- https://pinboard.in/u:toyg/t:python/ [ RSS ]
• Weblogic -- https://pinboard.in/u:toyg/t:weblogic/ [ RSS ]
• EPM -- https://pinboard.in/u:toyg/t:epm/ [ RSS ]
• OSX -- https://pinboard.in/u:toyg/t:osx [ RSS ]

Slides for "The EPM Diet" presentation at Infratects TopGun 2015

I usually have a lot of fun at our Top Gun event, which is basically the only conference in Europe entirely dedicated to Hyperion/EPM infrastructure topics. Infratects being the leading player in the European space for this particular niche, it makes sense to get the community together and have honest conversations about best practices.

In 2014 I presented a talk about securing EPM. In 2015, in keeping with the tradition of telling people to do stuff they'd rather not, I talked about keeping servers fit by following an EPM diet, aka a number of little maintenance tasks to be periodically repeated.

I guess next year it will have to be about paying your taxes on time, or calling your mum every week.

Hyperion Financial Management (HFM) 11.1.2.4.0 cannot be fully secured

(If you don't know what HFM is or does, look away now!)

If you work in the Oracle EPM space, you know that the long-awaited version 11.1.2.4.0 has finally been released, without much fanfare. For HFM, this release marks a major turning point: core services have been deeply modified in order to make them run on Unix/Linux, and ASP.Net was entirely removed from the stack, so that HFM can be officially supported on Exalytics system running Oracle Linux.

HFM is built on the Microsoft C++ stack, so it was an arduous task to make it portable without throwing away a decade of development efforts. To Oracle's credit, the mission was basically accomplished; however, *nix shops will likely not be too eager to deploy 11.1.2.4.0 in production environments just yet -- nor will traditional HFM customers. This release is rough, with quite a few functional bugs (taskflows cannot be displayed; the interface does not fully work in IE11; and so on), and unfortunately security was another casualty.

If you enable SSL for internal communication between EPM components, all services should encrypt their network traffic. In order to do that, they have to be manually configured, installing identity certificates required by the SSL protocol; this is a standard operation in Weblogic, and documented for other components in the Security Configuration Guide. Unfortunately, the HFM application server:

1. is not a Weblogic instance (at least not officially -- will clarify this later)
2. is not documented in the Guide yet.

Straight out of the gate we have a problem -- what certificate will HFM use? Nevermind, let's assume there is one hardcoded somewhere (shock, horror!). We simply enable full-SSL in Common Settings:

Then activate SSL support in the HFM Application Server configuration task:

And we can see in the EPM Registry that these settings have gone in:

We then configure the rest of the stack to use SSL (which is quite trivial, these days), increase logging here and there (good practice when enabling SSL, since it helps with troubleshooting), restart our services and... it all works! Great! Hold on, what is this in the HFM log?

[2015-02-06T01:11:05.087+00:00] [FM] [ERROR] [EPMHFM-65559] [oracle.FM.HSSUTIL.oracle.epm.fm.hssservice.RegistryWrapper] [tid: 10] [ecid: 0000KhSSAzEEgKYjLpqIOA1Kp1Ib000000,0] [SRC_CLASS: oracle.epm.fm.hssservice.RegistryWrapper] [SRC_METHOD: getRegistryProperty] Invalid property isSSL for HIT Registry component app24. [2015-02-06T01:11:05.132+00:00] [FM] [WARNING] [] [oracle.FM.HSX.SERVER.oracle.epm.fm.hsxserver.service.HsxServerServiceManager] [tid: 10] [ecid: 0000KhSSAzEEgKYjLpqIOA1Kp1Ib000000,0] [SRC_CLASS: oracle.epm.fm.hsxserver.service.HsxServerServiceManager] [SRC_METHOD: init] An error occurred retrieving property isSSL from EPM registry, using default value :false. [2015-02-06T01:11:05.141+00:00] [FM] [ERROR] [EPMHFM-65559] [oracle.FM.HSSUTIL.oracle.epm.fm.hssservice.RegistryWrapper] [tid: 10] [ecid: 0000KhSSAzEEgKYjLpqIOA1Kp1Ib000000,0] [SRC_CLASS: oracle.epm.fm.hssservice.RegistryWrapper] [SRC_METHOD: getRegistryProperty] Invalid property SSL_Port for HIT Registry component app24. [2015-02-06T01:11:05.141+00:00] [FM] [WARNING] [] [oracle.FM.HSX.SERVER.oracle.epm.fm.hsxserver.service.HsxServerServiceManager] [tid: 10] [ecid: 0000KhSSAzEEgKYjLpqIOA1Kp1Ib000000,0] [SRC_CLASS: oracle.epm.fm.hsxserver.service.HsxServerServiceManager] [SRC_METHOD: init] An error occurred retrieving property SSL_Port from EPM registry, using default value :9092.[...] [SRC_METHOD: init] isSSLEnabled :false

Ouch, it looks like HFM is just ignoring our settings. Old-hat EPM infrastructure hackers at this point are probably thinking "Hold on, I know those 'not found' properties; they're usually seen on web-app nodes!" and that's exactly it. The HFM server fancies itself a web-app, and looks for web-app properties in its registry node; but that node is actually a custom non-standard type with non-standard properties "isSslEnabled" and "ssl_port" (lowercase).

To verify that HFM was not encrypting anything, I dusted off my miserable Wireshark skills and logged some network traffic.

As you can see in the screenshot, data originating from the HFM application server and using ports in the range used by HFM, are sending data down the wire completely unencrypted. Malicious actors could hide somewhere in your company network and silently siphon away all your precious financial data with minimal effort. If you care about security, you'll likely want to give this HFM release a wide berth, at least until this bug is fixed.

Bonus: The Strange Case of The Masqueraded Weblogic

As I mentioned above, the HFM application server is not officially a Weblogic instance. So why is it trying to look up registry properties typical of Weblogic instances? Maybe because it is, in fact, a bastardised Weblogic. If we look at the startup parameters configured in Windows Registry, we find the following:

What's that? A Weblogic parameter? Indeed it is. And why is it there? By default, Weblogic will use the cryptographic extensions found in the JDK (or JRockit) in which is running. By setting "nojce" to True, you can tell Weblogic to actually use its internal implementation of such extensions (which were probably developed by BEA back in the days, when crypto support in the JDK was a bit shaky). If you are bastardising weblogic, I guess you can also replace some of these classes with your custom versions, which is probably what Oracle was trying to do here.

"Securing EPM" presentation at UKOUG Apps14

The slides for the "Securing EPM" presentation I've given at UKOUG Apps14 in Liverpool two weeks ago, are now available here. Merry Christmas!

Dash docset for Python 2.2.1 (i.e. Jython for Weblogic / Websphere)

I use Dash quite a bit, so I just spent a little bit of time creating a docset from Python 2.2.1 documentation. This old Python version matches the Jython implementation shipped with Oracle WebLogic ("WebLogic Scripting Tool", or WLST) and IBM WebSphere.

To install it in your Dash, just click on this link:dash-feed://https%3A%2F%2Fraw.githubusercontent.com%2Ftoyg%2Fpy221dashdocs%2Fmaster%2Ffeed.xml

The source script is in my GitHub repo, and you can manually download resulting packages on the Release page.

As tempting as it is, the idea to repackage webapp-specific documentation (e.g. for connect(), startEdit() etc) is a non-starter due to Oracle and IBM being quite trigger-happy with their copyright lawyers.