Memory Leak in node.js


Never, ever forget that node.js is single threaded and expects every callback to return to the event loop. Otherwise it will never be garbage collected.


This behavior was experienced in the Softwerkskammer platform. In order to show small avatars next to member’s names we load them from gravatar or take an uploaded pic. These are then binhexed and inlined in the page’s content. We started doing this every time the memberlist is requested. This is very slow and additionally causes unnecessary load to gravatar’s site. We then introduced a local cache using node-cache. I wanted to get rid of this caching and persist the gravatar along with other profile data in our DB.

Of course, we do not know, if a gravatar has been updated. Therefor we need to regularly check gravatar’s services.


We had code like that (no caching anymore):

function imageDataFromGravatar (url, callback) {
  request.get(url, function (error, response, body) {
  if (error) { return callback({image: null}); }

  var image = 'data:' + response.headers['content-type'] + ';base64,' + new Buffer(body).toString('base64');

  callback({image: image});

module.exports = {
  getImage: function (member, callback) {
    var url = this.avatarUrl(, 16);
    imageDataFromGravatar(url, function (data) {

The function “getImage” was called like that (for each member in a loop):

if (!member.getPersistedAvatarData()) {
  avatarProvider.getImage(member, function (imageData) {
    store.saveMember(member, callback);
} else {
  if (/*actuality check*/) {
    avatarProvider.getImage(member, function (imageData) {
      var oldAvatar = member.getPersistedAvatarData();
      if (member.getPersistedAvatarData() !== oldAvatar) {
        store.saveMember(member, function () { /* background op */ });

Do you spot the horror?

        store.saveMember(member, function () { /* background op */ });

I really thought, if I don’t give the “saveMember” function a callback method (which is expected by that method, it eventually calls a mongodb function that expects a callback), it just performs its stuff in the background. – *haha* background, *haha* background, *haha* background – today I am laughing at that. A few weeks ago I have been completely naiv.

So what did I do?

quite simple, changing the last few lines to:

if (/*actuality check*/) {
  avatarProvider.getImage(member, function (imageData) {
    return store.saveMember(member, callback); // never, ever "fork" stuff in node by not having return values *I AM IDIOT*
} else {

(please note the comment after the important line)

That’s it. I hope it helps you to not do the same mistakes.

P.S.: Finding this took me multiple days, spending lots of hours of effort.


Do you take it? – i.T.A.K.E.

I haven’t written an article for a long time. It’s time to come back.

During the last two days the i.T.A.K.E. unconference took place in Bucharest. I had the honor to be one of the presenters there. But I don’t want to speak about what I presented. Instead I want to thank the organizers and the visiting crowd that they created such an awesome event.


I felt really welcome and taken care of. I want to explicitly mention the organizers of Mozaic Works. They are a family that is open minded and open hearted and they let us be part of their family. The location was pleasant and – apart from minor problems (audio and video) – it went very smooth.


You people have been a great mix of experienced and fresh faces. Everyone has been very open and curious. I had many interesting and inspiring conversations about coding and team dynamics. It especially is worth going out with you in the evening.


In talks, I learned about a frontend developer’s perspective of the code (thanks @gion_13), found an easy way to explain “ports and adapters” (thanks @johan_alps and @sanlaville) and much more. But what really made my week was the keynote of @felienne. I usually do not like to listen to academics talking about software development. She changed my opinion.

Bottom Line

I am sure the event will happen again next year. It is definitely a “Not-To-Miss-Conference” if you are passionate about software craftsmanship and giving and taking.

Currently I am still sitting in my room in the conference hotel. A warm feeling is creeping up my belly when I think back of the last hours. Thank you Bucharest!


If it’s Crap, Call it Crap – not Legacy

Tonight we were discussing code quality issues. During this discussion one thing came up to my mind:

Everybody is calling old code “legacy”, and now legacy is used as a synonym for code of poor quality.

Why that?

My first guess is: If it was me who wrote the code, “legacy” sounds much nicer than saying “this code is crap”. “Crap” sounds harsh, brutal, nasty.

But there’s more to it. Many developers have to deal with existing code on a daily basis. Much code. Code that is running at customers’ sites. This code cannot be all crap. People use it, so it must be somehow valuable to them. We even earn money with selling it to new customers. – But it slowly kills us. It kills our ability to change it to modernize it.

Why bother?

Most of us are aware of the fact that today’s produced code will be tomorrow’s legacy. If we allow the use of the word “legacy” as a synonym for “crap” then we have a perfect excuse for producing crappy code. – We should care more of tomorrow’s legacy. We should be proud of yesterday’s work. Today, tomorrow, next year, in five years!

For example: We live in a legacy house, more than a hundred years old. And we are proud of it. We are proud of the people who have built it. Of course there are necessary refurbishments and modernizations, but performing them is usually possible with quite low effort. – I drive a car that is 20 years old. And I love it. – Both the house and the car are legacy, but not crap.

I have a Russian friend that possessed a 7 year old Lada. It was slightly legacy but complete crap. We once bought a brand new replacement part (a rear axle). It was cheap, but crap from the start. Of course the car was running most of the time, but we often had the feeling that we were lucky to reach our destination.


Let’s imagine a huge class hierarchy. Nowadays we know that in most cases huge hierarchies are not a good idea if you want to write maintainable code. Some years ago this knowledge was not so wide-spread. But the now so big hierarchy started small, believe it or not. So if you start building a hierarchy today you are potentially laying the foundation for tomorrow’s hell. And you should consider yourself a crap-maker, because you should know better than – let’s say – 10 years ago. The knowledge that deep hierarchies are problematic and should be avoided is now a “best practice” and has been around for at least five years. This means that if you are now dealing with a deep hierarchy that is less than five years old, you can frankly call that crap.

If you even think of extending this hierarchy with more depth, that is also crap. No excuses. If a structure you have to deal with already is too complicated, adding more is not a way of producing legacy, it is just a horrible practice and it always has been. As soon as you sense the code you are working on is getting too complex, stop. Refactor or refurbish it. It will never get easier in the future. The more you add, the more difficult it will be.


Think – don’t just code!

Legacy Software – Why I love working with it!

For years I worked as a consultant and coach. Both on agile methods and coding. I guess I was of some help on a few occasions. At least that is what some customers stated. But in the end I always left them alone. I often felt relief when I left them. But on the other hand it was also like surrender. The software was not finished. Most software is never finished; development goes on and on and on and …

This is probably nothing new to you. Maybe you have similar experiences. But probably you belong to the majority of people in software development: employed developers.

Buried Under Tons of Code

Old code. Untested. Unreadable. Unloved.

Is this what you dreamed of when you started your career? – Probably not. You may have dreamt of creating new shiny software from scratch using great technologies and fancy tools.

But you don’t quit your job. Perhaps you are a masochist? – I don’t think so. There are many reasons to face the challenge working in such an awkward place:

  • Responsibility: You feel responsible for the product. Although it is not the “Big Fun”, it is your job to keep the thing alive and even add more features to it.
  • Challenge: You think it is much harder to work in a situation like yours than to do small projects from scratch. You know that every software will become legacy after a certain age or size.
  • Fear: You are afraid of being the one to produce the nightmare from scratch that you are now working on. For you it is easier to say: “I just clean up other people’s mess!”
  • Hope: You are just staying because you have hope that you will be amongst the chosen few to start the next product from scratch.

In my case, it is a mixture of the first two aspects. In addition I have a strong belief that every legacy product can be changed to the better so that further development will be possible for a long period of time. This work is not funny but deeply satisfying. In our current product everyone in the team is trying his best while working on the huge codebase. And you know what: The software and the code become better each day.

We are all learning. We are all improving ourselves. Maybe some of us will become masters in the end. Who knows…

No IDE: A New Experience

Last week, we were attending a small conference that also included some one-day workshops. I chose to dig a little bit into Node.js. The workshop was held by Golo Roden.

I must admit I’ve had prejudices against JavaScript. And so I chose this topic to gain new insights. Now I know a lot more about JavaScript, have more questions and I really wonder:

Is it possible to conduct large complicated projects without an IDE?

I started developing software more than a decade ago with Smalltalk. For those of you who do not know Smalltalk: There is no separation of the IDE and the product. You don’t even work with files. Have a look yourself: – The non-existence of files in the Smalltalk IDE will give you an impression of how I feel with files. About ten years ago I had my first encounter with a different language based on files: Java. – Wow, that was strange at first.

To you this may seem weird. Programming without files? – To me it has been the other way around.

And last Thursday I’ve been attending my first workshop without even an IDE. Only an editor and a command line. – And it worked well for me. At least I was able to type in all the code (or should I say text?) Golo showed on the screen and made only few typos. I was really happy to have a quite sophisticated editor capable of JavaScript: syntax highlighting, able to show brace coherence.

But: I really wonder whether this will scale.

  • How will this work with more than ten files?
  • How will you do refactorings?
  • How can you see relations?

To me one aspect – if not the most important one – of programming is to be able to navigate my code. I definitely do want to see the implementation of a method or function I call. I want to be able to find the potential callers of a function. All this typing is so distracting from the product I’m about to build.

  • I have to know what is already defined to be able to use it.
  • It’s a pain (and slow) to look it up by reading potentially concerned files or documentation.
  • I have to remember the filenames where stuff is defined.
  • I have to track everything inside my head – in detail!

Is this really a professional working style? – What do you think?

 Instead I’d like to be able
  • to see an overview of my code
  • to see the big picture

This is even hard in today’s IDEs. But closer than just an editor.

P.S.: Some googling pointed me to obviously working IDEs for JavaScript with more than syntax highlighting.

The Quality Of Software Development Cannot Be Measured

I often read about the cool sides of agile and lean and other process frameworks or methodologies: you can measure where you are and how good you are doing what you want to do.

Concepts like Velocity or Lead Time help you predict what you will probably be able to achieve in the near future. But this is only the easy part: pure arithmetics.

Stop dealing with the easy, focus the difficult.

Did I say easy? – Yes, and I mean it. The above concepts (velocity, lead time) are easy as soon as you are able to measure them. You need a few requirements like: definition of done, being able to track tasks, estimations, discipline. Given you have all the requirements you defined, deducting the velocity (or lead time) is straightforward, if not to say trivial.

So please stop talking or writing of these concepts as some great discovery! – We all know that one of the hard parts is to fulfill those requirements.

If you are in a situation where the requirements can easily be fulfilled, chapeau! – You are one of the lucky few whose environment is in a great condition.

Now, what is the hard part? – It’s the quality.

Quality is hard.

How can you know if your quality is right?

Can you give me a sound definition of what quality means for you?

Can you even give me a definition of quality that you share with a few others?

Can you give me a definition that you can share with your team, with your organization?

There should be a consensus on these definitions that is supported by everyone who is concerned with the outcomes. And then you have the even greater dilemma: How can you prove that a certain result matches your definition of quality?

If you measure quality it becomes – by definition – a quantity!

Quality measures are only indicators.

Measures always face the problem that – once defined – people try to match the measure, not the intention behind it. Wouldn’t it be great to work together with a shared intention of quality instead of simply fulfilling metrics and measures?

If you have few/many bugs this can mean you have good/bad software. It need not be so.

If you are able to build new features quickly/slowly this can mean that the inner quality of your product is ok/not ok.

Metrics and measures are useful as indicators. I find them much more useful as indicators for bad quality than for good quality. Never be too satisfied when the indicators show everything is fine, but really be worried if they indicate a potential problem. – But this is just based on my experience. I’ve just never experienced a project with bad indicators not having serious problems sooner or later, but I’ve seen the opposite. This goes along the lines of watermelon reporting, which you may have experienced, too.

Quality is not objective.

I remember when Apple released their first iPod shuffle. A German consumer review organization (Stiftung Warentest) downgraded the device because it didn’t have a display. It simply didn’t fit their quality criteria. Nowadays they downgrade Apple devices because you need proprietary software to fill them with music. This can be a turn-off for some people but I guess it is not for many of those who own one.

As you can see by these examples – there are many more, just think about it – it is all about meeting expectations: The expectations of the people that define the measures. If these expectations are unmet, they regard the quality as being poor and vice versa.

And in software development? – Do we really have “hard” measures for software quality? – Do we even agree on very trivial topics? – Just ask a few developers how many LOCs per method are ok and you’ll probably end up in a joyful discussion. – We don’t even have a standard definition of LOCs!

Who can define measures for software quality? Insiders of the project rather than outsiders? There is always a trade-off, but I would always prefer the trustful outsider over the insider. Especially in legacy products it’s been the insiders that have produced the software as it now is. Especially if the system is a mess right now, would you trust them to give you a suitable advice? They see more resistance than outsiders do and they are part of the system, which means they are part of the resistance.

Back up quality by simple measures.

The struggle we are in should be the struggle for quality. Establish few but critical measures and teach your mates the intention behind them. In the domains I usually crawl around these may include:

  • Method length (combined with e.g. cyclomatic complexity)
  • Class size (number of methods)
  • Module dependencies (especially: no circular dependencies)

Why those? – They give an early indication of when you are in trouble to lose control of your code. But if you only strive for superficially fulfilling them through technical changes, it will result in a shattered system and the result on quality will be minimal. Finally, trust your experience or a friendly stranger with more experience than you have right now to judge if the code feels alright. If you can’t trust your feelings, start working on improving your intuition. And please don’t contaminate your intuition with knowledge. Intuition means you are able to judge the quality of a given piece of software before you can tell why. The explanation of your judgement is based on knowledge.

So please don’t mix quality and quantity. If you really want quality, stop treating it as numbers!

Does this sound obvious? – Then take your time and think about it.