Hosting Numerry, or the adventures in finding a cloud provider

In its simplest form, numerry is a tool for scientists to quickly plot and visualise data. That’s my personal pain point, the frustration to be solved behind numerry. The first use case is doing this on a desktop. Yet I don’t want to build a graphical user interface (GUI) per operating system, or resort to Java or Qt. No. I want to learn modern web development, and that means building the GUI for the web, even if first it’s going to run locally.

This will allow me to embed numerry in a website and make it accessible without local installation. That part is going to be a lot of work, because it requires a proper website, a database, etc. At some point I’d like numerry to become a platform where users share their plots, results, and, why not, even calculations. And for this purpose, I need to host it somewhere. Self hosting

When I started real programming, I was in a studio in Paris, for a post-doc. There, I wanted to learn Python, and as I would at some point be looking for another job, I decided to buy a cheap second hand computer to host my blog, where I would document my existence and its applications, programming being indeed a part of it. It turned out to be OK, actually. I had a good internet connection with sufficient bandwidth not to be the bottleneck. This computer was only running my Django website and the resources were enough. Yet, at some point, I was bored with the maintenance and the noise, so I decided to host it on a French web provider, where I still have an account that hosts websites for friends and my personal emails. They don’t provide Clojure support though, so I’ve been looking at something else.

My conclusions are two-fold. First, depending on your time, needs and budget, you really have to try hard not to find a cloud provider that suits you. Second:

The main decision to take is using a platform as a service (Paas) or an infrastructure as a service (IaaS).

A PaaS lets you focus on your project: the deployments are easy, you have a ton of integrations, you don’t have to care about the maintenance of the whole thing, etc. They are often less flexible and more expensive, which is the concession you make. I started with that to see if numerry would fit. Heroku

Everybody talks about it, it’s the de-facto goto place. I enjoyed it while I was there. Hobby plans are free, to let you experiment. You can plug databases, logging and analysis services, continuous integration and deployment, it supports Clojure and even builds it for you, and much more. That’s pretty expedient I have to say. A proper database and a better machine would cost me around $14 a month, which is the budget I had in mind (I could also have stayed on the free plans).

What I saw though, is that all this automation was frustrating me. Numerry is a hobby project, it’s a mean to learn new cool things. Had it been a one-shot for a week end, sure, let’s put it there, that’s perfect for it. What I realised is that I want to learn the full stack, from the commit to the deployment, the database choice and the migrations, the monitoring of the application, etc. Sounds masochistic or exciting, your milage may vary.

Hence, no PaaS for numerry. Before looking at the IaaS, let me plug shamelessly another PaaS I discovered recently, Clever Cloud, which is run in Nantes by an awesome team; I’m going to use it next time I need a PaaS for a specific project.

Google Cloud Platform

The big disappointment… Looked awesome, the power and the prices of Google at the reach of my hand… until I discovered that it’s only for real business, kids, not for hobby, pass your way.

https://twitter.com/adrienhaxaire/status/712019622057414656

I didn’t get any response indeed. I didn’t even look at Amazon Web Services, I guess it’s the same approach, and anyway they don’t inspire me.

Digital Ocean

When listening to an episode of FLOSS weekly, Randal gave a $10 coupon for Digital Ocean. My cousin, who’s building websites for a living, had already told me about it, so I gave it a shot. And I’ve not looked back since then. I’m not going to praise them too much, though, but here’s what I like:

Many available data centers, including two in Amsterdam.
Good prices.
Always one CPU per server (they call them droplets).
A clean and fluid interface, I'm still impressed about it.
A huge documentation, with forums, tutorials, etc. I've found everything I was looking for.
An awesome feel of freedom: create a droplet, play/experiment with it by installing whatever comes to your mind, then turn it off or destroy it.

I’ve been toying around for one month now, and I trust it enough to land numerry there. I’ve experimented different databases, different distros, different droplets, and many more, and I really like it. This made me rethink the architecture of numerry, how to distribute the services, which database to choose, what talks to what and how, etc.

I know I sign in for a lot of extra work when configuring the droplets, the reverse proxies, the firewalls, the java versions and whatnot, but that’s the reason why I went for an IaaS solution: to experience it, to roll up my sleeves and work at the bare metal level, to understand fully what’s going on and what’s needed to get a (mid-)simple website to run in the wild.

As I mentioned before, this is the choice I made for this project. If a friend comes to me to have me building him a small website, I’ll go for a PaaS solution, probably Clever Cloud. Right now, I’m proud of understanding how a reverse proxy works and how to configure it, how to deploy a Clojure application that runs as a Linux service, how to talk internally to droplets, and much much more.

One final word. It’s just the beginning and I’ve got a lot to learn and to do, but guess what, I have all the time I want to figure this out, numerry is an experiment, I have no deadlines but the ones I impose myself. They are not expressed in time, but in skills and knowledge I want to get.