06 February 2020

Announcing the AutoEPM Patch Tracker

UPDATE: This service has been discontinued; this post is left as historical reference. If you need this sort of thing, contact me on LinkedIn.

With the recent release of EPM 11.2, I thought it would be a good time to make public a tool I've built and used "behind the scenes" for quite some time, the AutoEPM Patch Tracker.

It's a patch tracker focused on EPM products, automatically aggregating updates and pointing to readmes, hopefully a bit faster to peruse than My Oracle Support. It also recommends Critical Product Updates that can be applied to the EPM stack, and the most common "standard" updates (only for 11.1.2.4 at the moment). With a free account via LinkedIn sign-on, all lists can be downloaded in CSV, JSON, and XML (Atom).

It's only a small showcase of what we can do at AutoEPM by Targlet. If you'd like to find out how to integrate this level of automation and intelligence to your EPM enviroments, feel free to reach out! I'll be at OpenWorld London next week as well, always happy to have a chat.

10 January 2020

Oracle Hyperion EPM 11.2 - first impressions

Last month, Oracle finally released the long-awaited on-premises release of EPM 11.2, rumoured to be the last hurrah for the venerable suite. After a couple of test installations, I have a few thoughts from an infrastructure perspective.

First, the bad news:

  • This is very much a .0 release, in many ways the first since 11.1.2.0 (the last time so much of the infrastructure had substantially changed). It is rough in quite a few places, and configuration is not for the faint of heart.
  • I have encountered what, to me, looks like a blocking bug in multi-server configurations. I filed a support request with Oracle and will see how it goes. At the moment I simply wouldn't be able to recommend it in situations where high-availability is necessary.
  • Even after configuration, it is harder to operate by non-techies than in the past. Even the "start" and "stop" links provided, which used to be ok for single-machine use at least, are not sufficient anymore - some extra scripting will be necessary to make it useable even in the most basic scenarios.
  • Documentation has not fully caught up with 12c, particularly in the area of additional configuration tasks for Financial Close Manager (which was already pretty lacking). Some of the additions, to deal with the now-necessary Repository Creation Utility, are ambiguous.
  • Weblogic seems hungrier for memory than in previous releases. This might be due to 12c, or to the fact that it's now using Java 8 under the hood - I had already measured an increase in this area when replacing Java 6 with 7 on 11.1.2.4.
  • Essbase seems to have been shipped almost unmodified from the latest 11.1.2.4 version. Among other things, this makes it currently impossible to do a full-SSL configuration if Essbase is in the picture. It's disappointing, particularly considering how Essbase customers are typically among the few who often require fully-encrypted solutions.
  • The documentation seems to state that running any "Configure Database" task more than once will end in failure. I suspect this is true only for certain components, but I have not had the time yet to explore this area. This makes it even harder than before to migrate databases after configuration, so make sure you take design choices that will stand the test of time.
  • EPMA lives! Well, not really - the product is gone, but somehow it's still avilable in the Provisioning screen and you can assign roles for it. 🤷🏻‍♂️
  • This is the end of the line for IIS stalwarts. Using IIS as front-end webserver will simply not work (according to docs).
  • It's not particularly clear whether Oracle 18c and 19c are officially supported as database. In practice they seem to work (after all, they are really 12.2.0.2 and .03 rebranded), but it would be nice if the matrix was clarified.

But there are also some good news!

  • The products themselves are basically the same, with the occasional extra feature. Users shouldn't need any substantial retraining, with the notable exception of any EPMA or Interactive Reporting power-users still around.
  • The 12c stack, in theory, unlocks a number of attractive possibilities for system administrators and implementors - containers, no-downtime patching, and so on. It will take some time to fully unpack how EPM can take advantage of these goodies.
  • Recent Windows Server releases are supported.
  • In my test, everything seems to work even with the free editions or Oracle and SQLServer (XE 18c and Express 2017), which is nice for consultants and other professionals.
  • Both 12c and Java 8 support modern crypto standards, which should make things less awkward when risk-assessing and pentesting.
  • Java 8 brings a lot of improvements to the language and the ecosystem. Whole classes of problems have been removed (MaxPermGen, anyone?), which should make services more reliable and performant. Custom integrations will also be easier and faster to develop and run.

In the past, I would have expected such a release to be followed by a quick stream of fixes, necessary to make it actually fit for production use. We will have to see if it is going to be the case here, considering how long it took to debut and the overall tone of the Oracle roadmap (mostly cloud-focused). Should Oracle really commit to improving the on-premise, this release would soon prove itself as a substantial improvement over 11.1.2.4.

If you'd like to know more and discuss your options in the EPM space, make sure to get in touch with us.

18 June 2019

How to dump macOS security-rights database

macOS handles some security items in a custom database, which may or may not be SQLite. The official way to interact with such database, outside of Objective-C, is the /usr/bin/security utility, with the parameter authorizationdb and specifying the required operation on the class of rights, e.g. security authorizationdb read system.login.console

Unfortunately, there is no way (that I could find) to simply list all classes. I was trying to uninstall something that might have had references in that db, but I wasn't sure about the class it might be registered under, so I wanted to dump them all.

Luckily I found what looks looks like a comprehensive list of rights. After a quick scraping job with Python, I had a list that I could use like this:

cat osxrights.txt | xargs -I % sh -c 'sudo security authorizationdb read %' | grep -B 10 myAnnoyingItemToRemove

... and the bundle wasn't anywhere, so I could just chuck it.

(Note: had the item been found, I would have had to dump the whole security class to a .plist file with security authorizationdb read the.class > my.plist , edited the file to remove it, then write it back to the db with security authorizationdb write the.class < my.plist )

14 March 2019

How to remove all bars in fullscreen mode in Firefox on MacOS/OSX ("kiosk mode")

Procedure

If you want to remove all toolbars from Firefox in fullscreen mode on Mac, this is the procedure:

  1. Type about:config: in the location bar to open advanced settings. (If it is the first time you do it, you might have to click "confirm" or something like that in the scary screen that comes up first.)
  2. search for full-screen-api.allow-trusted-requests-only and double-click the line that is found, so the value changes to false
  3. Bring up the Bookmark Toolbar if you don't have it (menu View -> Toolbars -> Bookmark Toolbar)
  4. Right-click on empty space in the Bookmark Toolbar, and select New Bookmark...
  5. In the Name field, type something like "Enter Fullscreen" (it doesn't really matter, just don't leave it blank)
  6. In the Location field, enter the following line:
    javascript:document.getElementsByTagName('html')[0].mozRequestFullScreen();void(0)
  7. Click Add to save the bookmark.
Now, when you click on that bookmark, Firefox will go "real fullscreen", hiding all toolbars. To get them back, you will have to press ESC.

Now, there is a bit of a snag. That preference in about:config: is set to true by default for a reason: it is a bit unsafe to allow any website to use fullscreen-enabling javascript. For this reason I would suggest to only do this for limited periods, then switch it back once you're done.

Long-winded explanation

FF has a preference, browser.fullscreen.autohide, that in theory should govern the behaviour of toolbars when going fullscreen. It is set to true by default, meaning toolbars should disappear. However, for some reason this property has (almost) always failed to work on MacOS/OSX builds of Firefox.

There used to be a few extensions that corrected that behaviour, but they've all gone away when Firefox switched to WebExtension APIs, which basically sandboxed extensions more rigidly and blocked them from changing the browser interface as deeply as they could before. This was the price for the dramatic performance improvement seen in Firefox 57+.

This workaround hence relies on the new Fullscreen WebAPI, which is supposed to be a new, standard way to enable fullscreen via Javascript on any browser (well, any browser that implements it - just a few at the moment). However, because of security concerns, this API is locked down by default; so, in order to use it, we first have to relax checks as described above. Once that is done, we can launch a bit of JS from a bookmark that triggers "real" fullscreen.

Credit to this AskDifferent post where I first found the trick.

11 February 2019

How to customise Jetty embedded in Spark Java framework

EDIT: I uploaded a full working example, ready to be customized, on github. If you just want a working solution, go there. Otherwise, keep reading for the explanation.

When it comes to Java microframeworks for API development, I've been using Spark for some time in a couple of different projects. For most casual needs, it works out of the box; however, there are circumstances where the "easy" options for configuring the embedded Jetty instance, are simply not enough. In that case, you have to take control and basically replace the instance with one that you completely control.

Due to how Jetty works, there are multiple moving parts: the server takes care of things like thread pools, the socket is responsible for SSL and other low-level protocol stuff, and the handler is where you do high-level middleware (changing headers etc). This means that you might need several factories to take care of all these. The good news is that Spark is granular enough that you can (more or less) limit yourself to what you need. The main pattern is: there will be a factory for each element, with a create() method returning an interface; you implement the interface and swap out the default factory with your own. However, these factories at the moment (Spark 2.8) are somewhat nested, so depending on where you need to work, you might have to replace a bunch of classes before you hit the point you're interested in.

The main entry point is EmbeddedServers.add(identifier, serverFactory). This is what you'll call from the main() method you use for all the post() and get() configuration directives. It is important that EmbeddedServers should appear right at the top, before any other configuration directive, otherwise Spark will use its default factory. It should look more or less like this:

EmbeddedServers.add(
     EmbeddedServers.Identifiers.JETTY, 
     new MyWonderfulServerFactory());

For the identifier, we re-use the default one because otherwise Spark will take over again. Nothing to do there.

The server factory is where real work begins. Your class (or lambda) needs to implement the spark.embeddedserver.EmbeddedServerFactory interface, which has one method:

public EmbeddedServer create(
            Routes routeMatcher, 
            StaticFilesConfiguration staticFilesConfiguration, 
            ExceptionMapper exceptionMapper, 
            boolean hasMultipleHandler)

As a starter, you can copy the content of spark.embeddedserver.jetty.EmbeddedJettyFactory as-is. You don't need constructors (unless you want them). Strictly speaking you don't need withThreadPool() and withHttpOnly() methods either, but I suggest you keep them anyway; just change the return type to match MyWonderfulServerFactory.

Create() does three things:

  1. Initializing the route matcher, which you probably don't want to touch;
  2. Initializing the Jetty handler for that matcher, which you may want to configure for things like header manipulation and other middleware;
  3. Creating the embedded server, which is likely what you are after.

I will assume we want to tweak n.3. The main place where to pay attention is this line at the end of create():

return (new EmbeddedJettyServer(this.serverFactory, handler)
        ).withThreadPool(this.threadPool);

This will return the default configuration, and we don't want that. So we change it to something like:

return (new MyWonderfulEmbeddedServer(handler)
        ).withThreadPool(this.threadPool);

Now we need a MyWonderfulEmbeddedServer class, which should implement spark.embeddedserver.EmbeddedServer. Again, as a starting point, you can copy spark.embeddedserver.jetty.EmbeddedJettyServer, and change the return types to match. I also suggest you get rid of the factory parameter in constructor and the related field, which adds a bit of unnecessary complexity. That factory is actually used only in one place:

if (this.threadPool == null) {
    this.server = this.serverFactory.create(
       maxThreads, minThreads, threadIdleTimeoutMillis);
} else {
    this.server = this.serverFactory.create(this.threadPool);
}

Which you can replace with the following (straight from spark.embeddedserver.jetty.JettyServer):

if (this.threadPool == null) {
    if (maxThreads > 0) {
       int min = minThreads > 0 ? minThreads : 8;
       int idleTimeout = threadIdleTimeoutMillis > 0 ? threadIdleTimeoutMillis : '\uea60';
       server = new Server(new QueuedThreadPool(maxThreads, min, idleTimeout));
    } else {
       server = new Server();
    }
} else {
    this.server = threadPool != null ? new Server(threadPool) : new Server();
}

It's a bunch of stuff related to the amount of threads and timeouts. If that's what you were trying to configure, btw, this is where you can do it. To be honest, I believe Spark developers intended that the "proper" way, to do that particular customisation, would be to keep the factory parameter as it is, implement your own alternative to the badly-named JettyServer class -- it should be something like JettyThreadConfigFactory, really -- which implements the similarly badly-named JettyServerFactory, and pass it to the constructor in MyWonderfulServerFactory.create().

In my case, though, I was after an SSL customisation, and for that I had to swap out yet another factory, spark.embeddedserver.jetty.SocketConnectorFactory, mentioned in the second half of this block:

ServerConnector connector;
if (sslStores == null) {
    connector = SocketConnectorFactory.createSocketConnector(
         this.server, host, port);
} else {
    connector = SocketConnectorFactory.createSecureSocketConnector(
         this.server, host, port, sslStores);
}

In this case there is no interface, you can just extend the existing class with what you need. I wanted to enforce a particular set of SSL ciphers and protocols, so I overrode createSecureSocketConnector() adding the following bits:

sslContextFactory.setExcludeProtocols("SSLv3", "SSLv2", "TLSv1.2");
// first we clear existing exclusions
sslContextFactory.setExcludeCipherSuites(new String[]{});
// then we re-add what we need
sslContextFactory.setIncludeCipherSuites(bigArrayOfCipherNames);

And that's it. Now you know how to instantiate your own Jetty instance for Spark. It's a bit convoluted, and hopefully a "spark 3" will give us a better architecture to work with, one day. In the meantime, this is how you can do it.