dcsimg

Summer Lovin’ Erlang

Tell me more, tell me more...

Last week I put my Erlang flag in the ground as it were. I was expecting a raft of flaming comments. Did not happen. I doubt there was a collective head nod from my vast audience, and I wonder why so quiet? Perhaps everyone is on vacation, or getting ready for vacation and have put off reading my insightful column for when they take a vacation. Speaking of vacations and beaches, that includes me. Weekly columns are still required, however. I have set aside some beach time to continue reading Programming Erlang: Software for a Concurrent World by Joe Armstrong. I have not yet, however, brought myself to bring my Eee web-book to the beach to play with some code. Bright sun not withstanding, I fear the union of silicon, both refined and shoreline, will not be a happy affair. Although this Asus photo might seem to indicate otherwise.

If you have a chance to read the above mentioned book, you will notice that programming with Erlang is different. As I mentioned previously, Erlang is a declarative language and as such should give “procedural” programmers headaches. Not to worry though since once you get your head around the functional/declarative nature of the language it starts to make incredibly good sense. Time for a confession. In years past I had implemented and used the Prolog language. Erlang has some features found in Prolog, but removed some of the more esoteric aspects i.e. backtracking. Based on my experience, as a trained procedural dude (mostly C and Fortran), once I “got it” Prolog became a very powerful language. Indeed, I found I could easily craft a program in less lines of code, less time, all the while focusing more on my application and less on the machine aspects. Let me be clear (should I say “declare”?) about my intention, I am exploring Erlang as a multi-core/multi-node language. I am not evangelizing the language nor am I standing on a soapbox telling you to use Erlang. I’ll let the code speak for itself as I explore Erlang in future columns. For now, I want to present some of the “big picture” features that make Erlang attractive to me.


One of the things I find interesting is that Erlang was designed for concurrent operation from the very beginning (circa 1988). That is, before multi-core, before Linux clusters, and maybe even before some of you were born. Erlang was developed by Ericsson to support telecom applications where massive concurrency (e.g. tracking phone calls) and having good support for fault tolerance was a must. Since no language could meet these constraints, a decision was made to create a new language. Good ideas from existing languages were incorporated as needed and thus Erlang was born.


Jumping ahead to today, the concurrency feature seems like a real good idea given the whole multi-core thing. Unlike more traditional languages, Erlang concurrency is a fundamental part of the language. And, most importantly, it is simple to use. A typical Erlang program consists of many separate processes. Each process owns its own memory and can only communicate with other processes though message passing. There is no shared memory in Erlang. Therefore, there are no locks, semaphores, etc. that are required to support “shared state” between processes. This design is liberating. Each process essentially takes care of itself and basically does not give a damn about other processes (this is not strictly true, but it makes for good drama). In addition, Erlang processes are independent, light weight, and owned by the program and not the operating system. Therefore, creating/destroying Erlang processes are very fast as is communication between processes. Remember the only way a process communicates with other processes is through a message — like you and me.

Working with processes is very simple. There are three basic operations: spawn, send operator, and a receive. For example, a message in Erlang is sent as follows (Note some language features have been omitted for clarity and % is a comment.)

%Process One
Pid = spawn(fun some_function)
Pid!some_message

%Process Two (some_function)
receive
% do something
end

Process One spawns some_function and a Pid (unique process ID) is returned. It then sends some_message to Process Two using Pid. Meanwhile, Process Two, will wait for a message and act accordingly when it is received. That pretty much describes the basic Erlang communication scheme. Of course, communications can be more sophisticated than the above, but the basic flow is essentially the same.

Erlang was designed for fine grained concurrency. Programs can easily use hundreds and even thousands of processes. Think of them as a kind-of subroutine thread in procedural languages. Only, they have not global variables. (Not to worry, I’ll talk about global data in the future). Erlang processes can live on a single processor, multiple cores, or multiple machines. This feature is transparent to the user. For instance, a concurrent Erlang program could be easily written on say and Eee PC at the beach, then run on an eight way server back at the office. No code changes are required and in most cases the programs will run faster. The program developed on the Eee PC can also be massively concurrent supporting many thousands of processes if necessary. Processes can be spawned on remote nodes as easily as the same node. Of course there is some hostname and security issues that need to be addressed, but the spawn,send, and receive dynamics are exactly the same.

Interestingly, and Erlang program is in some ways like an MPI (Message Passing Interface) program running across cluster nodes. Memory is not shared between nodes, only messages. Indeed, MPI programs can run on a single multi-core node and maintain the same private process model. Of course, messages may be passed though shared memory, but each process can only touch its own memory. Once difference between Erlang concurrency and MPI concurrency is the granularity. An MPI program uses messages to designate specific user selected concurrent parts of the program. In an Erlang program everything is concurrent (or as concurrent as possible). Thus, in Erlang the concurrency is always there if you need it because that is how you wrote the program.

It’s time to hit the beach, so I’ll share one final thought for this week. Erlang is often referred to as a “telecom language.” I think this is a mis-characterization. Of course, Ericsson designed Erlang for their needs, but it does not mean it can only be used as a “telecom” language — whatever that means. History is with me here. In case you forgot, UNIX and C were developed by AT&T to help manage a phone network. Now where did I put my speedo?

Comments on "Summer Lovin’ Erlang"

fporcher

Starting a colum about Erlang is welcome and a must at the time the software industry is indeed lacking sound and powerful development frameworks to address the full potential of multiple-core multiple-CPU computers.

Erlang is a full blown “pure” functional programming language strongly oriented towards concurrent programming.

Apart from the powerful and very expressive clause-oriented syntactic constructs used to define functions, Erlang retains nothing from the declarative aspects usually associated with the family of logic programming languages ; there is no backtacking mechanism, nor unification operator.

Indeed, Erlang native and powerful “match” operator is best described as a “binding” operator (tree-binding) that Lisp lovers have long known as the “let” construct.

There is no set, setq, or equivalent, and therefore no assignment operator per se : once bound, the value of an Erlang variable cannot be changed.

Along with message passing, this is precisely what makes Erlang so safe and easy to use for concurrent programming (ok, once the programmer has forgotten about idiosyncratic iterative constructs and acquainted himself with the more robust functional approach) : no shared memory, no shared state, and therefore no need for lock or semaphone.

Sort of parallel programming made easy !

As emphasized by Douglas, despite its roots in Telecom, Erlang should be seen as a general programming language and a platform of choice for today’s concurrent programming.

My feeling is that wide and successful Erlang adoption lies first of all in helping developpers to realize how easy the shift from iterative programming (OO or not) to functional programming.

Franck PORCHER, Ph.D.
fporcher@smartech.pf

Reply
smedda

Erlang is a “telecom language” because based on “asynchronous message passing interface”. Of course this avoid shared memory and thus locks, but it leads to duplicate information maintained in several processes, a need to keep it synchronized and a lot of crossing cases. Moreover lots of external interfaces like database access are not easy to use in an asynchronous way.

In term of efficiency, you do have lock as a process cannot deal with the following message until it has finished with the previous one. You lose time as well in process switching and because each process has first to find to which information the message it received applies to.

Reply
hilbertastronaut

I didn’t have much to say about your previous post because it’s obvious that a good runtime that supports high-level language features is the right way to go about developing on multicore.

However, one shouldn’t confuse a good runtime with the language built on top of it. The latter is mostly syntax and for skilled programmers, syntax is just religion ;-) Furthermore, one can build many different languages on top of the same runtime, which also can enable interoperation of the different languages’ data structures, and please the expert programmers who are particular about the syntax they use. The nonexpert programmers (who are primarily domain experts and not hackers) shouldn’t have to care about the syntax — they only want to “get things done” and are willing to endure any arcane and annoying syntax in order to have libraries that let them solve their problems with a minimum of coding time. (Perl, COBOL, APL, …)

Building a good runtime is hard and not as attractive for marketing as making a new language, hence it’s not done very much. Often the groups that do make an investment in a runtime do it to solve internal problems (e.g., “How do we support the current ecosystem of multiple development languages, while sharing common system and debugging facilities?”) which shouldn’t be visible to ordinary programmers. However, the really interesting features for parallelism and concurrency (e.g., load balancing, parallel garbage collection, reliability, debug traces, software transactional memory) are all in the runtime.

Reply
dmpase

As a hard core geek I would love to believe that the main thing required for a programming language to succeed is technical superiority — good syntax, good concepts/foundations, good run-time environment and good support. Unfortunately, as a former language designer for the HPC community my experience says otherwise.

I was co-author on a functional programming language (called “Flo”) for a company called Floating-Point Systems in the mid-80′s and co-author on the CRAFT Fortran extensions for Cray Research in the mid-90′s. Both languages were designed with the HPC community in mind, both had the full support of the companies involved, both had many person-years of effort behind them, and both were received enthusiastically by their community, at least for a while. Neither language lasted more than a few years and reasons cited for their demise were never technical merit.

Another functional programming language, Sisal, also experienced a similar fate. Many years of effort were poured into its implementation, a community at Lawrence Livermore National Lab used the language (at least apparently) with enthusiasm. High Performance Fortran (HPF) also rose and fell, as have many others.

Having a high-quality language, compiler and runtime environment with all the necessary tools for support, at least in each of these cases seemed to be a necessary condition but by itself it was not sufficient. It also had to somehow be accessible over a wide range of architectures and a long enough period of time to develop the programming skills necessary for wide-spread adoption. It needed to be taught in Universities and it needed to be perceived as something that would be around for a long enough time that substantial SW development in that language would not be perceived as money wasted.

I’m not saying this to dismiss Erlang – quite the opposite. I don’t know whether Erlang is the answer to our parallel programming needs or not, but I do know intimately the requirements of parallel programming and the merits of functional languages and Erlang sounds like it has potential.

If we assume that Erlang is the right combination of features and implementation, my question would be how do we make sure it doesn’t fail for some of these non-technical reasons? How do we make it happen? “Build it and they will come” is just wishful thinking.

Reply
dmpase

To my earlier comment I probably ought to add that IMHO Erlang needs to play nicely with existing software. I know that might add some ugliness to Erlang, but I think excluding access to existing code may create too much of a barrier for wide adoption.

When C came out it played in a different space than Fortran, but as it gained popularity eventually it needed to play nicely with Fortran. C++ was inherently compatible with C from the start. Java can interoperate with C/C++ and (even) Fortran.

There is a huge mass of existing software that, frankly, the community cannot afford to rewrite just because the language is different. Much of the software is potentially compatible, at least in concept, with functional programming ideals. For example, there are many algebraic functions such as SQRT, SIN, COS and such, that could be interlaced with Erlang relatively easily. On the other hand, much of the existing software deals with access to hardware and hardware inherently manages state, which is more of a problem for functional languages.

However this junction is managed, it seems clear that Erlang cannot simply live in its own world. Somehow it needs to play nicely in the sandbox with other software. Maybe it does already, I don’t know. Could someone who knows Erlang comment on this? (Dr. Eadline?)

Douglas Pase, PhD
douglas@pase.us

Reply
hilbertastronaut

smedda: Have you heard of Mnesia?

http://www.erlang.org/doc/apps/mnesia/index.html

Reply
bradadkins

It should also be noted that CouchDB looks very promising as a document oriented asynchronous database system. Written in Erlang.
http://incubator.apache.org/couchdb/

Reply

Awesome blog post.Thanks Again. Fantastic.

Reply

I simply could not depart your site prior to suggesting that I actually loved the usual information a person provide on your guests? Is gonna be again steadily to investigate cross-check new posts

Reply

I’m not sure exactly why but this weblog is loading incredibly slow for me. Is anyone else having this issue or is it a issue on my end? I’ll check back later and see if the problem still exists.

Reply

Today, while I was at work, my cousin stole my iPad and tested to see if it can survive a thirty foot drop, just so she can be a youtube sensation. My apple ipad is now broken and she has 83 views. I know this is entirely off topic but I had to share it with someone!

Reply

Here are some hyperlinks to sites that we link to due to the fact we think they may be worth visiting.

Reply

I really like your writing style, fantastic information, thankyou for putting up : D.

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>