on Aaron Swartz

I've lived through my fair share of internet lore in the last 15 years. Since 2001, Aaron's name kept popping up here and there; he belonged to that special pantheon of people whose genius is so clear, it makes other people despair that they will never, ever measure up to it. When I found out he was A FUCKING KID, well, it wasn't an easy day for my ego. His code was always, invariably fantastic; at a time when people were saying "Python will never match Java for raw performance", he (re)built Reddit to sustain inordinate amounts of traffic, and gave away the code for good measure. He kept contributing his flawless logic to umpteen projects, from Markdown to Django, helping them succeed with grace and selflessness. And then he started going really political, and damn, was he fighting all the good fights. A lot of people claim to be "talented" or be "good", but you could see he was the real deal. He was the sort of person I wanted to be if I only I could rewind my life 20 years and start again.

Back in the '90s, we used to think there was such a thing as "internet culture". We used to think the online world was free, the 'net community would have been immune to the evils of offline corruption, information wanted to be free, "they" couldn't fight disruption, progress was inevitable. 

Lies, all of them. 

We are losing this social revolution like our fathers lost in the '60s and '70s. Established players have made clear, over and over, that they will crush our lofty ideals as soon as we hit their wallets. They will ruin our lives, bankrupt us, force us into exile. Aaron is the last casualty of this counter-revolution. Sooner or later there will be a Vienna Congress moment, indeed it almost happened last month. 

Aaron was the best we could be, and even him couldn't bear the pain of living while fighting the good fight in this corrupt world. How are we supposed to cope? The small ones with simple jobs, normal families, little talent, broken dreams, and our own baggage of bad life choices and mediocrity -- how are we supposed to still believe?

Ah fuck. What a sad day.

Why I've never really liked the Facebook API

The other day, I got an email from Virgin Media stating that my connection had been "upgraded to 100Mb/s". I went to a bunch of speed-testing websites, and reported speeds were indeed much higher than in the past. I was tempted to brag about it on Facebook then I remembered that, last time I did something similar, I was humbled by a bunch of Dutch friends with "big pipes". I wondered what sort of speed they reported then, so I went to Facebook to search for that old status. And that's where my problems started.

The standard FB search UI failed to return anything even vaguely related, as usual. So I started googling for apps that would allow me to search my previous posts, and found a few which just wanted to gather all my personal data (on FB -- you don't say!). Then I found that you can actually request a complete download of all your data from FB (under Settings) and launched the request, but it looked like it would take a long time (for the record, I finally got it about 24 hours later). So I thought "hey, surely I can work with the FB API". How naive of me!

There is, in fact, a straightforward API call to get your statuses: /me/statuses. By default, it will return some 25 records, and links to further paginated results. Except pagination is ridiculously buggy: after the first 50 records, it will just return a blank page. If you try to use the limit parameter, it will return a maximum of 100 records per page, and again it will stop after the second page (i.e. max 200 results, which it's actually 199 because everybody knows "there are only two hard things in computer science"). Time-based parameters (until, since) didn't seem to work at all. Using wrappers rather than direct calls didn't seem to make any difference. Being very late, I gave up and went to sleep.

A day later, still incredulous and obviously fairly frustrated, I googled harder and finally found a relevant question on StackOverflow, which pointed to a Facebook bug about pagination. As the bug says, you can work around the problem by always using offset rather than relying on 'next' and 'previous' links returned in JSON responses. I verified and that's actually the case. By now, my export was available for download anyway. You can imagine how happy I am (not).

Lessons learnt from this whole debacle:

• The unofficial facebook-sdk for python doesn't work with Python 3. There is an experimental fork with very limited testing (i.e. it passes 2to3 and that's it).
• the json module in Python 3 Standard Library, as used by facebook-sdk, chokes on Facebook data. Don't even ask me how I found out. Trying with a more up-to-date version from the original upstream doesn't help. There is a Python 3 fork which didn't help. Juggling between json.load and json.loads didn't seem to help, and I didn't want to rip the guts out of facebook-sdk in fear of dropping compatibility with 2.x (although I cringed at times: using "file" as variable name? Really?). No wonder @kennethreitz rolled his own JSON parser in Requests.
• facebook-sdk should probably be rewritten from scratch in Python 3 using Requests. Not that I'll ever do it.
• After so many years and botched revamps, the Facebook API is still terrible. For something reportedly so essential to "2.0" internet infrastructure, and with so many uber-smart people on their payroll, the whole thing still feels incredibly hackish.

Public therapy

I just experienced my first real professional failure in almost two years of consulting for my current employer. I'm not saying I've always been perfect before, but it's the first time a customer basically told me to give up and go home.

As tempted as I am to blame the damn tool (which I had never seen before in my life, and unsurprisingly refused to do my bidding), the hard truth is that:

• I failed to properly and fully "sniff out" customer requirements in advance. This should have been a huge red flag, but I thought I was good enough to just deal with it. Lesson 1: there is a reason hubris is a cardinal sin.
• On finding myself in trouble, I kept hacking at the problem for days when I should have just taken a step back straight away. I kept googling for a magic bullet, when I should have admitted that I did not know how the product was supposed to work, should have gone back to studying from first principles, and should have built a local proof-of-concept before attempting a real-world deployment. Despite being completely honest with the customer at all times, I ended up over-promising and under-delivering, which is the exact opposite of what I always try to do. Lesson 2: if you find your axe is blunt, hitting faster and from all directions will not compensate; just stop and sharpen up, there is no shame in it.
• Because of this "just a little hack" attitude, deep down I was not fully committed and concentrated on the problem. I kept assuming that solving a second main task would "make up" for failure on the first one. Unfortunately, this second task depended in part on other people, who also failed to deliver on time. Lesson 3: deus ex-machina is a literary device, not an action plan and certainly not a plan B. Also Lesson 4: be brutally honest with yourself at all times.

Obviously, I'm not happy today. Regardless of the actual task at hand, I failed at thinking strategically and being self-aware, and at my age this should not happen; I'm pretty sure I've learnt all these lessons when I was 22, and still managed to forget them. I hope this little recap will help me focus... my next engagement looks like a slam-dunk and I owe to myself to make it so.

"Mark of the Ninja" does not work under VmWare Fusion 5

I installed "Mark of the Ninja" in an XP image under VmWare Fusion 5, hoping the game would have been "simple enough" to run. The image has 4 cores and 4GB of RAM, so plenty of resources. Unfortunately, it looks like it will just crash XP soon after launch. Turning 3D support off, the game doesn't even start. Ah well, i'll have to dig out some old laptop...

Free disk space by removing TimeMachine local snapshots

Mac OSX TimeMachine's "local snapshots" are basically backup files stored on your local hard-disk (as opposed to regular ones stored on the external TimeMachine server). This is a useful feature for people who travel a lot, so they can revert files to recent versions even when they are not connected to their home TimeMachine.

Unfortunately, this feature tends to take a lot of disk space whenever you deal with big files (like movies etc). In some cases, you might want to temporarily disable it in order to claim back a few GigaBytes. Note that your regular TimeMachine backups will NOT be affected; you'll just lose the ability to revert to recent (i.e. from last week) versions without being connected to the external TimeMachine.

TimeMachine is managed from the command line with the tmutil command. You can type man tmutil to see all options. To disable the snapshots and delete those big files (which are found under /.MobileBackups, by the way), use "sudo tmutil disablelocal". Once snapshots are purged, you can restart the feature (if you need it) with "sudo tmutil enablelocal" (yeah, not exactly rocket science).

In my case, this procedure freed some 139 GB on a 512 GB disk. Not bad!