eFortunes 0.1: a distributed, scalable and fault-tolerant fortunes server developed in Erlang

Well, after naming this post in this way I think it could be a good idea to explain what this is about… :)

As the result of working in my Master Thesis, I developed an small “ad-hoc” framework for programming simple and so-so-small web applications using Erlang, since a requirement for doing my Thesis was using distributed functional technology… and web applications are not usually developed with this kind of tools. Well, I know that with ErlyWeb this assertion could be wrong in a matter of months ;) , but when I started my work (a year ago) the situation was so much different from now.

In the other hand, my fellow worker Javier Muñoz had recently developed a so much practical script (“Erlang On Rails”, as he called it) for easily start with the development of Erlang applications, without getting lost (and bored) with several commonly needed issues, such as the application definition file, the compilation process or the launcher script. On this post you can read more about “Erlang On Rails”, as it was told by Javi some days ago.

With this background on mind, I felt myself encouraged to develop an small web application that let me to carry on learning about the Erlang world, now that my Thesis is (at last) over… and I thinked about doing an “absurd” (well, this is subjective :) ) fortunes server could be a good choice, since I could learn with it more about so many issues about Erlang I haven’t learnt yet, or get a deeper knowledge about issues I’ve already know. For instance:

  • Distributed persistence of data (fortunes) using Mnesia, a Erlang-based objectual DBMS.
  • Persistence into a relational DBMS (such as PostgreSQL) using the ODBC API provided by the Erlang/OTP platform. (Nowadays, it seems Mnesia is not suitable for storing large amounts of data)
  • Developing reliable and fault-tolerance applications on Erlang according the design principles of the supervision tree, as suggested on the Erlang/OTP documentation at erlang.org. (Well, sometimes fortunes are a really-critical information… don’t you think?)
  • Designing of web applications with Erlang using Yaws, a light web server developed on Erlang which is quickly growing on these days, adding more and more capabilities with each version that is out (such as AJAX or haXe support).
  • Testing several frameworks I’ve recently discovered on the web, and that I wasn’t able on my Thesis because of a matter of time: Yatsy, ErlyDB, ErlTL, ErlyWeb
  • And so on…

In fact, all these points could be abbreviated as “learning more about Erlang and having an workbench for testing all those issues I’d like”. And if, while learning in this way, I get a distributed, scalable and fault-tolerant fortunes server for being able to safely store jokes and spontaneus comments… “better than better” XD

At last, I’d like to thank my Master Thesis’s director (Víctor M. Gulías) for letting me to license my work under the terms of the GNU General Public License, in order to let everyone to get the source of eFortunes and use it as needed.

Now I’ll end this post by telling the current features of eFortunes 0.1, the TODO list (this one could change, of course) for eFortunes 0.2 and some screenshots of it. And, of course… you can download the source code if you want from here:

efortunes-0.1.tar.gz

eFortunes 0.1 features:

  • List, create, edit or remove fortunes from server
  • Underlying distributed persistence layer based on the Mnesia DBMS
  • Easy access to the services offered by efortunes by using a simple web interface (powered by yaws)
  • Internationalization (i18n) support for the implementation of the web interface

eFortunes 0.2 TODO list:

  • Bug fixing ;)
  • Add the capability of importing/exporting from/to regular fortunes files
  • Add search interface for making possible to look for fortunes according some specific criteria
  • Improve the web interface ( well… this is a must ;) )
  • Add an user profiles management tool (currently, only an user admin/admin exists)
  • Create an standalone, deployable daemon of efortunes (maybe, yaws should be embedded into an erlang application)
  • Documentation and code-cleaning

eFortunes screenshots (sorry, only spanish text on them):

List all the fortunes

Edit a fortune Deployment of eFortunes

Stressing servers with Tsung

Tsung is a testing tool used to stress servers and see how they perform under high load conditions. It’s designed to work both with the HTTP/HTTPS and the Jabber protocols, and it seems that stressing PostgreSQL servers will be another feature in a near future (still experimental). Tsung is able, for instance, to simulate hundreds of users from a single CPU working as clients of a client-server aplication, in order to stress the server with high load and see how it works under such those contidions.

A very interesting issue about Tsung is that it was written using the Erlang language, which was designed for being used on some kind of environments where real-time issues, concurrency, fault tolerance and distributed computing are required features. On this way, Tsung uses the Erlang lightweight processes to simulate each hipothetical user, and that’s why Tsung is able to “create” a so impressive amount of simultaneous users from a single CPU.

Another interesting feature is that, since Erlang was designed for distributed environments, Tsung is designed to take advantage of this when designing the “stressing architecture” for your server: you can have only a client stressing a server, of course, but you also have the chance of having a cluster of Erlang nodes working together for being able to stress even more the “defenceless” server. And that’s the reason I wrote this post: today I was benchmarking an application I started some time ago, and I was really impressed by the way Tsung manages this task… and the ease of getting it working just from downloading the sources from its web site.

Unfortunately, documentation about Tsung it’s not too much (but enough), so might be you spend some time trying to understand how you can configure it, and how to use it… but, when you already know that issues, it’s so easy to use it and so impressive to see the results… especially if you are stressing an application running over the yaws web server, which is also developed using Erlang and is able to work with lots of simultaneous requests, as you can see here.

In conclusion, if you are currently looking for a tool to test your server, I’d suggest you to take a look into the Tsung web site and give it a try… especially if you have several computers connected through a LAN, and you can use them as a “stressing cluster”. I think you’ll like it.

Enjoy it!