The Diligent Developer

The Diligent Developer Framework

2024-01-31T00:00:00+01:00

What makes a great programmer, and how can you become one? This is an important question that many authors have tried to answer.

On one hand, we have great books like The Pragmatic Programmer, or A Philosophy of Software Design, which address this question heads on. They will make an experienced software developer nod in agreement from cover to cover, but they won't resonate as much with a junior developer.

On the other hand, there is no shortage of courses and tutorials on specific programming languages and frameworks. But after following some of these tutorials, you may still find that something is missing and be lost on how to continue improving.

The Diligent Developer framework is an attempt to provide practical guidelines to assess and improve your skills to become a better programmer.

The 6 skills

I have divided the skills that a developer should master into 6 categories:

Coding & problem-solving
System design and architecture
Programming language
Third-party components
Communication
Environment

Before going deeper on what each of these skills mean, let me explain the reasoning behind this list. The list is comprehensive and orthogonal: mastering all 6 skills will make you a great programmer, but being good at any of these skills doesn't automatically make you good at the others.

I have seen great problem solvers limited by their lack of understanding of the programming language they where using, I've seen proficient programmers unable to use a relational database to their advantage. It's common for great developers overall to end up working on the wrong problem due to some miscommunication with managers or colleagues.

Let's go deeper on each of these skills.

1. Coding & problem-solving

Coding is the translation of your thoughts into code, and the related tasks that that requires. It includes writing new code, fixing a compilation error, debugging an issue, writing a unit test. Coding is what you do with a problem that you can fit in your head, when you focus on a specific task.

Some people may think that coding is not that important, that it is the kind of thing junior developers do, while senior ones focus on system architecture and other high-level topics.

Quite the opposite, you need to be a great coder in order to become a great software engineer, and framing coding and architecture as competing skills is completely misguided. There are plenty of great coders that have lead big projects and organizations without compromising on their coding skills, e.g. Peter Norvig, George Hotz or DHH.

Others might argue the opposite, that coding is all there is to software development, and the rest is just fluff. This may be the case for extremely talented developers who take for granted a bunch of other skills, but rarely constitutes a good piece of advice for the rest of us.

The best way to become a better coder is to deliberately practice on problems of the right difficulty: neither too hard nor too easy. I find the Advent of Code to be a great exercise, but to make the most of it, you need to become aware of your process. Iterate on your code to make it as simple as possible, think about what took you so long, why a given bug was difficult to debug, why some program you wrote worked flawlessly and another required a lot of back and forth, and compare your solution with others.

The good thing about improving your coding and problem-solving skills is that solutions that you come up with will be simpler, which will make those problems more manageable, and will allow you to tackle bigger problems.

2. System design and architecture

Coding and problem-solving can only get you so far. Eventually, you will face a problem that doesn't fit in your head, and you will need to break it down into smaller problems.

At a small scale, this is what you do when you write a function or a class and choose its parameters, attributes, and the way they interact.

At larger scales, you need to think about larger components, how they are implemented, how they communicate, how they are deployed, how they are monitored, by whom they are developed.

A bad design introduces unnecessary complexity and can have all kinds of deleterious effects: it can affect performance by adding bottlenecks, make the system error-prone by making it difficult to test, slow development by making the setup of the development environment difficult.

We developers have a tendency to tackle imaginary problems. When thinking about software architecture, we tend to think about the future: scalability, high availability, teams of developers.

However, a good architecture should be as simple as possible to get the job done, and should leave room for future changes.

To become a better system designer, there are two things you need to do. First, you need to think deeply of the implications of your design decisions. When there is a bug, an outage, when a developer is blocked, you need to think hard about how your design contributed to that outcome.

Second, you need to learn from others' experience. Blog posts are best in my opinion. Read why someone thinks microservices are stupid, and how someone runs a company on 3,000 microservices, how someone deploys 10 times a day to production, why someone stopped using X because it was unreliable. With time, you will start to see patterns, and you will be able to make better decisions.

3. Programming language

Whenever you are coding, you are using a programming language. And which one you use is a big deal.

An argument could be made that it's not important which programming language you use. Twitter was built on Ruby, Instagram on Python, WhatsApp on Erlang, so... does it really matter what programming language you use? It seems you can accomplish mostly anything with any programming language. Why not just pick one and be done with it?

The truth is that the programming language you use does matter. WhatsApp chose Erlang because it's designed to be fault-tolerant and distributed, Twitter and Instagram were built on Ruby on Rails and Django, the web frameworks that revolutionized web development in the late 2000s.

A programming language is not only its syntax, it's the ecosystem and the community. It's the libraries, the frameworks, the tools, the conventions. As stated almost 20 years ago by Paul Graham in The Python Paradox, a programming language should get you excited by itself.

There are two paths to improve on this skill: learn more about the programming language you are using, and learn more programming languages.

It's difficult to overstate how deep you can go into a programming language. Mastering Python, JavaScript, Java, or any other language, is a life-long journey. You may start learning the syntax, then there's the standard library and third party libraries. You can go a long way with just this knowledge. But then there are the conventions, the right way to do things, the idioms, the advanced features: concurrency, generics, metaprogramming; and then there's the internals: the implementation of the language, the compiler, the virtual machine, the garbage collector.

Some developers are deeply bonded to one programming language. They learned to code with it, grew to think in that language, and it makes them very productive. Sometimes, this attachment is not so healthy, and thinking of trying new languages fosters fears of being unproductive, feeling dumb, or wasting time.

It's important not to confuse learning a programming language with learning to program, which generally go hand in hand the first time. Most of the coding skill is transferable from one language to another, and learning new programming paradigms can be a low-hanging fruit for improving your craft.

4. Third-party components

We love to code, but we can't code everything (there are some exceptions).

Databases, caches or message queues, and libraries that implement encryption, compression, graphics are clear contenders for not being coded from scratch.

Knowing how to use a relational database or a message queue is hugely beneficial for you as a developer. For one, it removes a ton of responsibility from you. In a hundred years, I couldn't develop an equivalent of PostgreSQL, so it is great that it exists and I can use it for free.

Apart from filling a concrete need in our system, third-party components bring other benefits. Being exposed to great tools that solve difficult problems also serves as an example to follow in your own coding. This is specially true for open source tools.

Good and sane decisions in naming, API design, backwards-compatibility, etc. can serve as an inspiration for your own work.

But there's a world of difference between knowing that a tool exists, and knowing how to use it. Phrases like "just put a cache in front of that service" or "just store it in a database" are often thrown around, but properly implementing them is much harder.

Even though each tool is different, I've also found that learning how to use them is also a transferable skill. Learning to test and benchmark different configurations, navigating the documentation, finding help online, prototyping before going all in are things that you can practice.

Similar to third-party components, we have services. Sentry, Datadog, AWS, Mailgun, Stripe are undoubtedly useful services, but they are also black boxes. They come with great benefits, but the skill of using them is not that transferable and doesn't contribute as much to making you a better developer.

If you are used to relying on managed databases or clouds, I suggest you install some of those components locally, and play around with them. Redis, PostgreSQL, ClickHouse, SQLite, NATS are all great and inspiring projects.

5. Communication

When you code, you are communicating with the machine, but you are also communicating with other developers. You communicate through your code, with your choice of names for variables, functions, classes, and also through your comments.

Apart from the code, you also communicate through commit messages, bug reports, issue comments, instant messages and emails. And beyond written communication, you surely have conversations, meetings or presentations.

Being effective at these forms of communication is crucial to be a good developer. It's not uncommon for great developers to be outstanding bloggers, science fiction writers or best-selling authors. Counter to popular belief, good programming skills correlate more with good communication skills than with math talent (see The Programmer's Brain).

A crucial side effect of improving your communication skills is that of improving your thinking. Has it ever happened to you that in the process of writing a detailed StackOverflow question or a question to some colleague you found the solution to the issue yourself? It's not a coincidence. By making the effort to explain the issue to someone else, you are forced to get a clearer understanding of the problem, and this often leads to the solution.

Similarly, by splitting each commit message into logical chunks, with a description of the changes introduced, you can get a better understanding of what you are doing, and you can also help your colleagues understand your code.

Practice is crucial to become a better communicator, and there's a lot you can do. Trying to write the perfect email, documentation page or commit message can be blocking and frustrating. It's better to practice and try to improve gradually, getting out of your comfort zone bit by bit. Explain the last bug you fixed to your hairdresser, write a blog post about this thought you had, spend 2 more minutes on that commit message.

Communication plays a central role in your work as a software engineer and it requires you to be conscientious just like for any other skill.

6. Environment

Programming is one of the most purely intellectual activities you can do, but it doesn't happen in a void. Mastering the tools around your craft can greatly improve your productivity.

This environment includes your code editor or IDE, the debugger, shell, version control software, password manager, to-do app, operating system and any other tool you use to get your job done. It also includes your physical setting: keyboard, mouse, screen, desk, chair, and the room you are in. You could even fit your diet, exercise and sleep habits in this category if you wish.

All of these things can make a big difference in your productivity, specially for the worse. You will not get a huge productivity boost by upgrading your keyboard or getting a comfortable chair, but you will get a huge productivity hit if you develop RSI or back pain.

Tweaking and improving your environment is the topic on this framework that is more prone to fetishization. It can be exciting to build your own keyboards and learn a new layout every 3 months, but you must not kid yourself into thinking that this will make you a better developer.

Still, there are some things that are worth investing time in. Same as with coding, you need to be aware of your process and realize when your environment is the bottleneck.

Conclusion

We developers take hundreds or thousands of decisions every day, big or small. I hope this framework allows you to zoom out and think of your craft as a whole, hopefully providing some guidance in your journey.

Where to go from here

I write a newsletter in which I share my thoughts on software development, productivity, entrepreneurship and more. Subscribe below if you want to receive it.

Nothing changing is a change in itself

2023-12-13T00:00:00+01:00

I'm in a video call with 15 other software engineers, managers, devops. Two of them are from my company, the rest are from the client and their systems integrator.

The system has been down for 3 days, and they are not happy. Tens of thousands of dollars are lost every day of downtime.

They pinged us 3 days ago saying there was some issue, and after checking our monitoring we responded with an everything looks fine on our side.

During the following days the issue escalated more and more, and their conclusion was that it was our fault.

Their thinking was simple: we haven't changed anything in 1 month, no new code, no new deployments. Therefore, it must be someone else's fault.

I am still pretty sure that our systems are working fine, but it is difficult to get out of the situation. I ask for proof. Have you added some debug messages that show that the request is being sent, but no response is being received? That is in my mind the simplest way to see where the program gets hanged.

Their answer is that this is not even necessary, because this is a proxy microservice that only calls our service and then returns. Now that, to me, is a red flag. In three days of outage, the least you could do is confirm your suspicion of where the problem is; specially if it takes 5 minutes to confirm, before pulling 15 people into a meeting.

I still don't believe the problem is ours, so I ask to see the code. My hope is to find out that their proxy microservice is in fact doing something else, and that is what times out.

They screen-share the code. It turns out it actually is very simple: a TypeScript implementation of a gRPC API which forwarded all requests to our remote server. Nothing else.

And as the client's VP stresses once and again, they haven't changed the service for 1.5 months, so it can't be their fault.

That finally gets me thinking... 1.5 months? The project started a few months ago, and I bet this is the longest time the service had been running without a restart.

Now, there is a reason why restarting the service might help. This is a microservice that acts as a proxy, and I know gRPC and JavaScript don't work extremely well together, it is not one of the best supported languages. After running for a long time, processing millions of requests, some answered, some timed out, some streaming that may never be closed, etc. it may happen that some buffer got filled up somewhere or the list of open connections grew too large or something, and then the service stops answering any request.

The bad thing is that it would look like the problem is that the external service is not answering, the good thing is that a simple restart would fix the issue.

Have you tried to restart the service?

No. They restart the service immediately, and everything is working again.

I cheered, but not everyone was as happy as me.

Now this is not only a funny story, but there are some things we can learn.

When you face a bug or an incident, it is important to explore by your own means as much as you can what the issue may be. In this case: they could have just restarted the service and added debug messages. Adding other people in the loop always adds delays, and you should only do it when you are blocked or it takes a huge amount of time to proceed by yourself.
Be careful with microservices architectures in large environments. One of the reasons why the service wasn't restarted is that the developer didn't have access to a restart button. They need to contact the devops team in order to get a restart, or a new deployment going. This makes sense to some extent, but it can add too much friction to just debug the simplest of issues.
Be careful with simple and stupid microservices. We need to call an external API? Let's add a simple API proxy microservice. There may be reasons to do this, but in 99% of the cases I would suggest you use a library for this, and not a microservice. The code may look so simple that you think it is completely robust. But running a separate microservice for this adds a bunch of complexity: more network hops, separate logs, a single bottleneck for all other microservices using the API. So you should provide a real benefit to you application in exchange of all this addeed complexity, and a dumb API proxy doesn't.

Spaced repetition really works

2023-09-13T00:00:00+02:00

Spaced repetition is the practice of learning with flashcards reviewed at increasingly longer intervals.

The logic behind it is that when you are first exposed to a piece of information, you are prone to forget it quickly, but revisiting it you will remember it for longer periods, and thus it's not necessary to review it as often.

This approach is widely used to learn new languages, and multiple scientific studies have proven it to be extremely effective.

This is all cool, but I was a bit of a skeptic. The reason is that people learning with Anki were generally extremely motivated and were spending a lot of time creating their decks, and these are the kind of people who are good at learning, with or without spaced repetition.

I needed proof, subjective proof, and I found it.

Last year, my friend Maria wanted to get her Spanish nationality, for which she had to take a test. The test has a pool of 300 questions from which 30 are chosen, and you are only allowed 3 mistakes. Some questions are extremely easy, but others are difficult even for a Spaniard. They ask about some facts such as who wrote some book, or what is someone known for.

The exam was due in one month, and Maria was pretty scared. She had been studying for weeks, but there were some questions she kept getting wrong, and it was hard to go through the whole list of 300 questions, names got mixed in her head.

I suggested giving spaced repetition a try.

I extracted a CSV from the PDF, and created an Anki deck which I later loaded on her phone with AnkiDroid, and she started reviewing the deck on her phone, going through the questions on her spare time.

The result was incredible. Of course, she aced the test. But not only that, in one or two weeks all her fear had faded. She was learning the whole deck, and she knew it.

This experience proved two things to me:

Spaced repetition works. The science is right.
You don't need to create your own decks. As long as cards are atomic, have the right context and are shown in the right order, there is no extra effort a learner needs to do.

This was my aha moment, and that's when I registered the domain python.cards.

Not all programmers realize how beneficial it is to know their language better. It's not only about coding faster or saving some Google searches, it's about boosting your confidence and having a solid understanding of your tool. It's about reaching higher levels of proficiency.

Experienced programmers master their language by being exposed to it for years, but there is a shortcut: spaced repetition.

Why solving the Advent of Code is a great exercise

2023-08-22T00:00:00+02:00

The Advent of Code is a yearly challenge that consists of 25 programming problems, one each day, from December 1st to December 25th. Hundreds of thousands of programmers participate each year, and use it as a way to improve their coding skills, learn new programming languages or just for fun.

I participated for the first time in 2020, having programmed in Python for almost a decade. I solved all the problems on the day they were published, didn't have excessive trouble with any of them, had fun, and went on with my life.

Three or four months later, I came across the Pytudes repository by Peter Norvig and my mind was blown. In case you don't know him (I didn't at the time), he is a legend in the programming world and has been the director of Google Research for many years, among other things.

The point is his solutions are brief, elegant, and use a ton of Python features that I didn't know about.

I was so baffled that I revisited all the problems, and worked on them, one by one, trying to understand his approach, and the Python features he used, and identifying the gaps in my knowledge that prevented me from solving the problems in a similar way.

What does Peter Norvig know, that I don't, that prevents me from writing a solution like his?

— me, on loop on my head

This was an inflection point for me. At the end of 2021, I participated again in the Advent of Code, but this time I focused not on solving the problems, but on how I solved them. I ran a small competition with a friend, trying to see who found the most elegant solution for each problem.

I'm not exactly proud of the outcome, it landed on the "hackish" side more than on the "elegant" side. E.g., for day 8:

digits = lambda t: "abcefg cf acdeg acdfg bcdf abdfg abdefg acf abcdefg abcdfg".translate(str.maketrans(dict(zip("abcdefg", t)))).split()

fsfs = lambda x: frozenset(map(frozenset, x))
PERMS = {fsfs(digits(p)): mapt(set, digits(p)) for p in permutations("abcdefg")}
nums = lambda ds, qs: [PERMS[fsfs(ds)].index(set(q)) for q in qs]

in8: List[int] = data(8, atoms)

def day8_1(rows):
    return sum(n in [1, 4, 7, 8] for row in rows for n in nums(row[:10], row[11:]))

def day8_2(rows):
    return sum(n * 10**(3-i) for row in rows for i, n in enumerate(nums(row[:10], row[11:])))

Anyway, by this time, whenever I read a Python code I had written more than a year ago, I would immediately spot things that could be improved and that I would not do in the same way. It feels great to see how much you improved over time.

For the 2022 edition, I went one step further and recorded myself solving some of the problems, and uploaded the videos to YouTube. This also proved to be a great exercise, because it made me focus 100% on the problem I was solving, and also because I had to be deliberate about what I was doing.

In case you are curious, you can check my code at this repository on GitHub, which has all my solutions and can judge for yourself how good or bad they are.

Mentoring with the Advent of Code

I've been mentoring a few developers over the last year, helping them improve their coding and software design skills. I have found that the Advent of Code problems are a great subject of study, even more than the typical Leetcode problems.

Approaching the problems deliberately, trying to find the "best" solution, and not just "a" solution, is a great exercise. When you have given your best, it is the best time to come in and give you feedback on how to improve: how to simplify the algorithm, make better use of the language features, improve the readability, etc.

My collection of scientific and technical books

2018-12-16T12:00:00+01:00

What technical books would you buy if you had $1000 to spend? This is the question I asked myself when I learned that my Ph.D. scholarship included some money destined to buy material that I had not spent. After a few days of research, I selected a list of more than 30 books.

It's been a few months now, and I've had time to read some of the books and leaf through the rest. Here you have my selection, with a brief comment on each book.

Bioinformatics algorithms

A wonderful book, which has just come out in its third edition. It explains in a didactic and entertaining way the fundamental algorithms of different bioinformatics fields. The book, available at my university's library, helped me in preparing the Bioinformatics contest 2018, and was an indispensable acquisition.

Physics for Scientists and Engineers

The classic book of physics, with chapters devoted to mechanics, thermodynamics, electromagnetism, relativity... I have always liked physics, and in high school I tried to learn as much as I could. This book will help me refresh and come back, from time to time, to whatever matters I may be interested on.

Calculus

I have often commented with some friends from university that we should have paid more attention to the courses’ bibliography in college. Studying based on books would have allowed us to return years later to those same books, which will always be better than the notes we might have taken. This book is the first step to correct that mistake.

Introduction to algorithms

The bible of algorithmics. A book that I consulted extensively to prepare programming competitions during my university years, which is much more than an introduction to algorithms and data structures.

Understanding machine learning

This book is an introduction to the theoretical foundations of machine learning. A very well explained book that has been useful to me to understand some methods I have used in my thesis.

Molecular Biology of the Cell

Although I am almost a doctor in biomedicine (bioinformatics more specifically), my knowledge on biology some years ago was rather limited. For me, the first chapters of this book were very useful to get in touch with the matter, and I have used it as a reference book for years.

Principles of biochemistry

The bible of biochemistry, a book that has everything I will never need to know about this matter.

Lewin's genes XII

A book on genetics that I did not know before doing this search. It has good reviews and is a good complement to the previous books on biology and biochemistry.

The C Programming Language

C is probably the most influential programming language in history. This book, one of whose authors is the creator of C, is the classic manual of this language.

Expert C Programming

Although the previous book is the basic reference of C, a lot more is needed to write good C. This book contains dozens of tricks, good practices, etc. that help us do it better.

CODE: The Hidden Language of Computer Hardware and Software

Interesting and entertaining book that summarizes the history of computer science, of all the advances that have converged in the creation of computers.

Programming Collective Intelligence

How does Google choose relevant web pages for a given search query? How does Netflix know what movies we might like? How does Facebook know which friends to suggest? This book, which has a peculiar title, deals with the algorithms that have taken great relevance since the rise of the web.

Competitive Programming 3

A real gem, how useful it would have been when I was a student! The authors, true masters of programming contests, condense in this book years and years of training, practice and study. In it they deal with all the issues related to programming competitions: algorithms, data structures, implementation tricks, relevant theorems and mathematical properties, and much more.

Computer Networks

The bible of computer networks. The truth is that I did not know this book, but my colleague Pau recommended it to me; he always remembers a lesson from this book: we can not underestimate the bandwidth of a van full of hard drives.

Deep learning with Python

Deep learning, that is, machine learning based on neural networks of many layers, has been a revolution in recent years in the world of artificial intelligence. The advances have been so fast, that today it is feasible to use Python to train your models, and you can even do it at home if you buy a good GPU. This book, written by the author of Keras, one of the most popular deep learning frameworks, introduces us to this world, beginning with the theoretical foundation and soon getting hands on practice. It should also be noted that the book is very well written.

Effective C++

A book that I have consulted many times in PDF and that I wanted to have. With this book, thousands of programmers have improved their knowledge of C++.

Effective Modern C++

The necessary complement to the previous book. C++ has introduced major changes in recent years, and Scott Meyers summarizes the best of these new features. Especially interesting are the chapters discussing concurrent programming.

The Go Programming Language

Go is a very interesting programming language, which shines in terms of simplicity and concurrence. It is specially designed for projects on which several people work, long codes where conventions need to be followed, and efficiency is important. This book is a complete introduction to the language.

RTFM: Red Team Field Manual

RTFM (usually Read The Fucking Manual) is what my friend Alberto used to say when I asked him for the nth time about some option of the grep command. This book contains a catalog of hundreds of commands from a variety of themes: Windows, *nix, networks, web, tricks... Most of the commands are unknown to me, which makes this very a interesting source of information.

Command Line Kung Fu

A book which is less advanced than the previous one, but gives us a good base to start working effectively on the command line.

The Hitchhiker's Guide to Python

This guide to Python, written by the author of requests and other popular packages, rather than a guide on Python, is a guide on good practices in terms of project management, how to organize the code, manage dependencies, etc. This book may become obsolete in a few years, but it is relevant at the moment.

JavasScript: The Good Parts

This book, published in 2006, has been very influential in the resurgence of the JavaScript programming language. JavaScript, a language that had very bad reputation for a long time, has become one of the most used languages in recent years. In just 100 pages, the author is able to present the strengths of JavaScript, helping the programmer to avoid common pitfalls, and providing a solid base to build upon.

During the last years, however, new standards of the language have been released, and it would be great if the author published a new and updated edition of the book.

Working Effectively with Legacy Code

How should we handle old code bases? Fortunately, many people have encountered this situation previously and their experience helps the rest of us to face these challenges. This is one of the most relevant books regarding this topic.

Refactoring: Improving the Design of Existing Code

Refactoring is changing a code for another that does the same, but which is better in some way. This book explores this concept, and offers us dozens of examples and patterns to detect candidate codes to be refactored.

The book was published almost 20 years ago, and is specially focused on Java and the paradigm of object-oriented programming, however, it has timeless lessons that can be applied to all scenarios. Interestingly, a new edition has just been published that changes this approach.

Mastering Regular Expressions

Regular expressions are a whole world, and this is the book recommended by everyone to learn how to master them. I remember my friend Rafa who wrote infernal lines of Chinese, as he called it, which were able to detect complicated patterns. Obviously they weren’t Chinese, but regular expressions; I hope this book helps me improve in this skill.

Pro Git

For some years now, myself, and practically all programmers, use Git daily to manage the versions of our programs and projects. Git was revolutionary: a distributed version control system, with cryptographic guarantees, very fast... and free as in freedom. This book takes us beyond pull, add, commit, push, and unveils the great potential behind this tool.

Programming Pearls

This book, a couple of chapters of which I had already read from a PDF a few years ago, contains authentic programming pearls. The author explains stories and some challenges, and we join him in the exploration of various solutions that show us the beauty that simple problems can hide.

The Rust Programming Language

Rust is one of the most interesting programming languages released in recent years. It is revolutionary in the management of memory, making memory security a priority but without losing efficiency at runtime. This is the second edition of the official introduction to the language book. Although it is available online, for me it has always been easier to read a book such as this in paper.

The Contest Problem Book VII, VIII, IX

As the promoter of the Contrarellotge matemàtica (in Catalan), an online math contests site, I'm always looking for problems. These three books compile hundreds of math problems that have appeared in the American Mathematics Competitions.

Doing Science

A book with an atrocious cover which contains interesting recommendations and reflections regarding research and its communication. It discusses crucial matters such as data collection, statistical analysis, writing style, tables, figures... together with clear examples and providing historical context. A book that I recommend to anyone who is getting started in research.

The Elements of Style

A classic guide for writing English. First published more than 100 years ago, it compiles lots of recommendations on writing that are still fully relevant. The book has been recommended by hundreds of journalists, writers, editors… it’s an ideal complement to On Writing Well, which I read a few months ago, and will be great help for writing my thesis.

30 interesting commands for the Linux shell

2017-09-14T00:00:00+02:00

This article got pretty big on Hacker News, check the discussion there for some interesting comments.

These are 30 interesting commands and tips for the Linux shell that I have collected over the years.

1. Supervise command (run every 2s)

watch "ls -larth"

2. Kill program using one port

sudo fuser -k 8000/tcp

3. Limit memory usage for following commands

ulimit -Sv 1000       # 1000 KBs = 1 MB
ulimit -Sv unlimited  # Remove limit

4. Rename selected files using a regular expression

rename 's/\.bak$/.txt/' *.bak

5. Get full path of file

readlink -f file.txt

6. List contents of tar.gz and extract only one file

tar tf file.tgz
tar xf file.tgz static

7. List files by size

ls -lS

8. Nice trace route

mtr google.com

9. Find files tips

find . -size 20c             # By file size (20 bytes)
find . -name "*.gz" -delete  # Delete files
find . -exec echo {} \;      # One file by line
./file1
./file2
./file3
find . -exec echo {} \+      # All in the same line
./file1 ./file2 ./file3

10. Print text ad infinitum

yes
yes hello

11. Who is logged in?

12. Prepend line number

ls | nl

13. Grep with Perl like syntax (allows chars like \t)

grep -P "\t"

14. Cat backwards (starting from the end)

tac file

15. Check permissions of each directory to a file

It is useful to detect permissions errors, for example when configuring a web server.

namei -l /path/to/file.txt

16. Run command every time a file is modified

while inotifywait -e close_write document.tex
do
    make
done

17. Copy to clipboard

cat file.txt | xclip -selection clipboard

18. Spell and grammar check in Latex

detex file.tex | diction -bs

You may need to install the following: sudo apt-get install diction texlive-extra-utils.

19. Check resources' usage of command

/usr/bin/time -v ls

20. Randomize lines in file

cat file.txt | sort -R
cat file.txt | sort -R | head  # Pick a random sambple

# Even better (suggested by xearl in Hacker news):
shuf file.txt

21. Keep program running after leaving SSH session

If the program doesn't need any interaction:

nohup ./script.sh &

If you need to enter some input manually and then want to leave:

./script.sh
<Type any input you want>
<Ctrl-Z>          # send process to sleep
jobs -l           # find out the job id
disown -h jobid   # disown job
bg                # continue running in the background

Of course, you can also use screen or tmux for this purpose.

22. Run a command for a limited time

timeout 10s ./script.sh

# Restart every 30 minutes
while true; do timeout 30m ./script.sh; done

23. Combine lines from two sorted files

comm file1 file2

Prints these three columns:

Lines unique to file1.
Lines unique to file2.
Lines both in file1 and file2.

With options -1, -2, -3, you can remove each of these columns.

24. Split long file in files with same number of lines

split -l LINES -d file.txt output_prefix

25. Flush swap partition

If a program eats too much memory, the swap can get filled with the rest of the memory and when you go back to normal, everything is slow. Just restart the swap partition to fix it:

sudo swapoff -a
sudo swapon -a

26. Fix ext4 file system with problems with its superblock

sudo fsck.ext4 -f -y /dev/sda1
sudo fsck.ext4 -v /dev/sda1
sudo mke2fs -n /dev/sda1
sudo e2fsck -n <first block number of previous list> /dev/sda1

27. Create empty file of given size

fallocate -l 1G test.img

28. Manipulate PDFs from the command line

To join, shuffle, select, etc. pdftk is a great tool:

pdftk *.pdf cat output all.pdf        # Join PDFs together
pdftk A=in.pdf cat A5 output out.pdf  # Extract page from PDF

You can also manipulate the content with cpdf:

cpdf -draft in.pdf -o out.pdf      # Remove images
cpdf -blacktext in.pdf -o out.pdf  # Convert all text to black color

29. Monitor the progress in terms of generated output

# Write random data, encode it in base64 and monitor how fast it
# is being sent to /dev/null
cat /dev/urandom | base64 | pv -lbri2 > /dev/null

# pv options:
#   -l,  lines
#   -b,  total counter
#   -r,  show rate
#   -i2, refresh every 2 seconds

30. Find packages that have a given file in Ubuntu

apt-file update
apt-file search dir/file.h

My first scientific paper: PMut reinvented

2017-09-07T00:00:00+02:00

Last July, my first scientific paper was published. Titled PMut. A web-based tool for the annotation of pathological variants on proteins, 2017 update, it is the result of more than two years work in my Ph.D. in bioinformatics. The paper has been published in the English journal Nucleic Acids Research, one of the main journals in the biomedicine field; specifically, it is part of the yearly web services special issue.

In this blog article, I will try to summarize the most relevant points of the paper, ommiting the technical details. If you have not read the article "Introduction to my research in bioinformatics", I recommend it as an introduction to the context of my research.

Classic PMut

PMut is a predictor of the pathology of protein mutations. That is, it is a computer program that, given a mutation in a protein, gives us a prediction of whether this mutation will cause a disease or it will be harmless.

This idea is not new, in fact the PMut predictor was created in 2005 and has been one of the most widely used predictors since then. However, during this last decade, the predictive methods have evolved dramatically and the number of mutations analyzed in the lab has increased by a factor of 10. Thus, I took the challenge of updating this predictor, and this scientific paper is the first result of my work.

The PyMut prediction engine

As a prelude to the development of a new predictor, we have developed the PyMut prediction engine in the form of a Python package. This software, publicly available, is capable of managing all the tasks related to pathology predictions: the calculation of numerical descriptors of mutations, the training of predictors, the evaluation of these, the generation of related charts, etc. Here are some examples of the charts we can get using PyMut:

Examples of charts generated using PyMut.

With the PyMut package, everyone can replicate on their computer the process of preparing, testing and using a predictor such as PMut.

The new PMut2017 predictor

Using the PyMut engine, and training with the SwissVar database, which contains 27,203 pathogenic mutations and 38,078 neutral mutations over more than 12,000 different proteins, we have trained the PMut2017 predictor.

Before giving the predictor for good, it is crucial to evaluate its reliability. In fact, the journal demands a very strict study of its precision, and for this reason we have followed 4 different approaches:

10-fold cross-validation. It consists of dividing the training data into 10 sets, training a predictor with 9 of them and evaluating with the one that has been excluded. We repeat this procedure 10 times, excluding a different set every time and averaging the results. This is the easiest way to obtain an approximation of the reliability of the predictor.
Blind test using new SwissVar data. We move to December 2015, and train a predictor with the data that was available back then in SwissVar. Then, we evaluate the 3,166 mutations that were added to SwissVar during 2016.
ClinVar data. Clinvar is another database of annotated mutations, significantly different from SwissVar. We have evaluated how good are the predictions of PMut2017 on the Clinvar mutations that are not present in SwissVar.
Study of specific genes. We compared how PMut2017 works for some specific genes, comparing the results with some of the most widely used predictors.

Comparison of PMut with other predictors. The most relevant metric is the Matthews Correlations Coefficient (MCC, last column), for which a higher number means better predictions. See the article to understand the details of the table.

All these assessments have allowed us to compare PMut2017 with the most accurate predictors and to verify that it is undoubtedly among the most powerful ones.

The PMut web portal

To make predictions accessible to the research community, we facilitate the use of PMut2017 from a web page open to everyone. Users can send a list of mutations through a form and obtain the corresponding predictions.

Screenshots of the web portal.

The vast majority of users ask for the impact of mutations on human proteins. In order to streamline these calculations, and since I do the Ph.D. in the Barcelona Supercomputing Center and we like to use the enormous supercomputer we have, we decided to pre-calculate some predictions. In fact not only some, but all the possible mutations in all human proteins, that is, a total of 725,596,928 mutations over 106,407 proteins. These predictions, stored in a MongoDB database, are immediately accessible without the need for any calculation.

Marenostrum 4, BSC's supercomputer, has a computing power of 11.1 Petaflops, that is to say, 11.1 × 10¹⁵ operations per second. Marenostrum 4 was released in July 2017; for my computations I used the late Marenostrum 3.

The PMut portal also offers the novel possibility of training specific predictors. This functionality is especially relevant to those researchers that work on one or more specific proteins. These researchers, who usually have more data than the public, can easily train a custom predictor and use it for their specific research.

Conclusion

In summary, in this work we have published a new predictor as accurate as the best predictors currently available. In addition, we share the code we used to train it, released as a Python package. Finally, we make all these functions accessible from a web portal where, for the first time, researchers can train —in a fully automatic way— tailor-made predictors for their studies.

Introduction to my research in bioinformatics

2017-05-08T12:00:00+02:00

Two and a half years ago I started my PhD in bioinformatics, a wide field of research closely related to biomedicine, which was totally unknown to me at that time. Biomedicine seeks the understanding (and treatment) of diseases at a molecular and biological scale, and bioinformatics adds the use of computational tools in this research. In this article I will make an introduction to my research, contextualizing it in the more general biomedical research field.

DNA, genes and proteins

All cells in our body (and any organism in general), contain huge molecules in the form of lengthy threads, which we call DNA. DNA is very important, because it acts like a big book of instructions for the development of living organisms.

Source: bbc.com/education

In the long thread which DNA is, there are pieces that are particularly important and are what we call genes. Genes are special because they contain some very specific information: the instructions to make proteins. Proteins, which can be imagined as tiny machines, are crucial to life because they perform all kinds of tasks: transporting components, accelerating chemical reactions, communicating signals, etc.

Example protein: myoglobin, which is responsible for transporting oxygen in muscle tissue.
Source: wikipedia.org

Mutations

As we all know thanks to many TV series, the DNA of each person is different. But how different is it? In fact, the DNA of individuals is the same in 99.9% of the sequence, especially in the genes: we all need the same proteins to perform the same functions and therefore our genes are almost identical. When there are differences, they are usually in the form of single point mutations: specific changes in the DNA sequence. These mutations, when they occur on a gene, may also mean that the protein also has a mutation.

When we talk about mutations, we are not talking about abstract or unknown entities, as we have techniques nowadays that can find the mutations in the DNA of people. Some tools that in recent years have been taking more and more significance are the DNA sequencing machines, which are able to “read” a person's DNA. For example, they can be used to detect differences between cancer cells and healthy cells from the same patient to try to understand how the disease is developing in a particular case.

DNA Sequencer: great strides over the past decade have meant that “reading” DNA is now be very affordable.
Source: illumina.com

Mutations and diseases

Hence, are mutations in proteins good or bad? The answer is that it depends. Most mutations are harmless, we could even say that they are good because they don’t cause any harm, they allow the variety of living organisms and are key to evolution. However, some mutations are also the root cause of many diseases, for example by modifying the shape of a protein and preventing the fulfillment of its functions.

Healthy red blood cells and aberrant red blood cells (“banana”-shaped) due to a mutation in hemoglobin.
Source: britannica.com

Finding out whether a mutation is bad or not is difficult and expensive. It is part of the work that many researchers do around the world, trying to discover the molecular-scale implications of mutations in the development of various diseases. These findings are usually made public and put together, and so there are databases of mutations classified as benign or pathological. These databases, which in some cases contain more than 100,000 mutations, are still too small and contain only a small fraction of all the mutations that can be found in patients. Therefore, we need more tools to differentiate between benign and pathological mutations...

Can computers help in any way?

Pathology computer predictors

Computer science, and machine learning more specifically, provide us with tools that can help us classify mutations as either pathological or benign. Even though these methods will not be as reliable as the evidence obtained in the lab, they can be a great help for other researchers, pointing at promising leads and helping in the interpretation of large amounts of data. The first pathology predictors were developed 15 years ago, they are computer programs based on various methods that are able to predict (with an accuracy that does'nt usually exceed the 90%, but that can get close in some cases) the pathology of mutations.

In 2004, a team of researchers from Barcelona (including the advisors of my thesis), developed PMut, one of the first predictors that has been widely used since. When I started my PhD, I was commissioned to put the predictor up-to-date. Much has happened since 2004, the mutations databases are 10 times larger, and great advances in predictive methods have been made. The “simple” task of updating the PMut predictor has lead to research on how to improve predictors, deepen in the understanding of mutations and the strengths and weaknesses of these predictors.

Soon I will publish a blog article explaining my first contributions to this field and the specific lines of my research.

A brief history of the Web... and web development

2016-10-13T12:00:00+02:00

1 of 42

This is a presentation I prepared to introduce a course on web programming. It reviews the evolution of the Web since its creation and how web development has changed over the years. The first website was published only 25 years ago, and it is nice to know the technology that allowed such a great expansion in order to understand web programming nowadays.

The story is very simplified and is made from my subjective point of view, giving more importance to technologies that I have personally used. Hope you like it!

The two aces paradox

2016-07-25T12:00:00+02:00

A few days ago my former programming teammate Ivan Geffner presented me a curious mathematical paradox that I will now explain. This problem appears in the book Reasoning about uncertainty by Joseph Halpern, a professor at Cornell University, where Ivan is doing his doctorate. Let's see what it is.

The game

A friend of ours presents us a game using four cards (from the Spanish deck): the ace of golds, ace of spades, the three and four of cups. Repeatedly, our friend shuffles these four cards and we pick two of them at random:

We randomly choose two cards from these four.

Then he offers a deal:

—Would you bet 1€ to having both aces? If you do have them, I will give you 4€.

As we are reasonable people, we will make some small probability calculations before entering the game. We note that there are six possible combinations of two cards, all equally probable:

The six possible choices of two cards

So, ultimately, we will win only 1 time in 6, and thus lose 6€ in order to win only 4€. So we tell our friend: I will not play!

Our friend keeps shuffling the four cards and handing us two of them. He looks at the cards we have discarded, and to make the game more attractive to us, he says:

Proposal #1. —I can see you at least have an ace, do you want to play now?

Show answer

Of the six initial combinations, we can dismiss the case where we have the three and the four, because we know that at least we have one ace. Then there are five possibilities:

In 1 case out of 5 we have two aces, so we have to spend 5€ to earn 4€. The answer is the same as before: I do not play!

Making a great effort, our friend keeps dealing and trying to offer us even better deals, like this one:

Proposal #2. —Now you have the ace of golds, do you want to play?

Show answer

Because we know that we have the ace of golds, there are only three possible combinations:

So every 3 times we bet, we will win once, so we will earn 4€ for every 3€ we bet! Then we tell our friend: I want to play!

Similarly, after dealing again, the friend suggests:

Proposal #3. —Now you have the ace of spades, wanna play?

Show answer

This case is equivalent to the one before, so we will also play! Having the ace of spades, there are only three possible choices:

The paradox

Finally, our friend reveals the paradox:

—When I tell you that you have the ace of golds or the ace of spades you decide to play. However, when I tell you that you have an ace (without specifying which one of them), you don't want to play. But it is obvious that if you have an ace you will have either the ace of golds or the ace of spades!

What is the explanation of this paradox?

Show answer

The truth is that we were a little hasty when we bet our savings to a game without a thorough understanding of its probabilities.

The key lies in the information our friend gives us, especially when we have two aces. If we have two aces, which one does he tell us that we have: the ace of golds or the ace of spades? Suppose he randomly says one of them and let's calculate the odds again.

When our friend tells us that we have the ace of golds, there are three possibilities:

We have the ace of golds and the three of cups, with probability $\frac16$.
We have the ace of golds and the four of cups, with probability $\frac16$.
We have both aces (probability $\frac16$), and our friend chose to tell us that we have the ace of golds (probability $\frac12$). The probability is $\frac16\cdot\frac12=\frac1{12}$.

Probabilities of all possible outcomes of the game

Thus the probability of having both aces when our friend tells us that we have the ace of golds is:

$$P=\frac{\frac1{12}}{\frac16 + \frac16 + \frac1{12}} = \frac{\frac1{12}}{\frac5{12}} = \frac15$$

In short, we'd better not play in any of the cases.

To understand this paradox we have had to think accurately about how the game develops, instead of calculating some isolated probabilities that do not match the real problem. Joseph Halpern presents in his book a mathematical framework with different measures of probability, conditioning, expectation, protocols, etc. and formalizes the concepts needed to tackle and solve these problems.

Learning Haskell

2016-06-15T12:00:00+02:00

Haskell is a functional programming language, very different to the programming languages I'm used to (C++, Python, JavaScript...). I've been hearing about Haskell for a while; people emphasize its elegance, its mathematical inspiration and foundation, and the fact that being based on such a different paradigm with respect to imperative programming it helps thinking about programming from a very different perspective. Python, Java, C++... are all languages that are incorporating concepts from functional programming to their latest versions. Therefore I was willing to learn some Haskell.

Haskell logo

Material

I began to look for books, tutorials, presentations, etc. For me, the best book for beginners is Learn You a Haskell for Great Good!. This presentation (in Spanish) by UJI professor Andrés Marzal is also very interesting and entertaining (in two hours and a half he starts presenting Haskell and ends up explaining some pretty advanced features), it also includes some great pdf notes. I also checked the book Real World Haskell, which I don't think is the best book to start with, but it surely has very interesting information if you want to develop larger projects.

Practice with... the Judge!

Once I was convinced of the benefits of Haskell, I wanted to start writing some code, but did not have any project in mind. That's when I thought of learning Haskell the same way I had learned to program in C++ many years ago: with the Judge.

The Judge with the classic traffic light: green or red depending on whether your program is correct or not.

The Judge is a platform to learn how to program, developed by professors Jordi Petit and Salvador Roura at UPC. This platform contains problems to be solved by programs that are sent to the same Judge, which automatically checks whether their execution is correct. The Judge is used in many UPC programming and algorithms courses and its use is open to the public, even more... it contains 33 problems of a Haskell programming course!

Thus, in recent weeks I have done these 33 problems :-). It has been a very fun experience and a great excercise to get familiar with the language and the development environment.

Haskell examples

Next I'll show some Haskell features to give an idea of its potential. These concepts are explained in more depth in Learn You a Haskell for Great Good!, here I will present them together with some code examples.

Working in Ubuntu, you can install haskell-platform (sudo apt-get install haskell-platform) and use the ghci interpreter. You can save the programs in files (e.g. prog.hs), defining in them the funtions you want. Then, these functions can be loaded (and reloaded) with ghci>: l to prog, which makes them available to be used and tested.

Let's see which are these interesting Haskell functionalities...

Recursion

Haskell, as a functional programming language, doesn't have loops: no while, no for. One way to implement what could be done using loops is through recursion. An example with the factorial function:

$$factorial(n) = n! = n\cdot (n-1)\cdots2\cdot 1$$

Recursively, we could define it as:

$$\begin{cases} factorial(0) = 1 \\ factorial(n) = n\cdot factorial(n-1) \end{cases}$$

And in Haskell, it could be implemented as follows:

factorial :: Integer -> Integer
factorial 0 = 1
factorial n = n * factorial (n-1)

The first line is the signature of function, which specifies the type of the parameter and result. In our case, we use Integer, which are integer numbers of arbitrary length.

Lists

Many programs need to work with lists of values, and Haskell offers us many ways to do it comfortably. For example, let's see some ways to get even numbers between 0 and 20:

Prelude> [2,4..20]
[2,4,6,8,10,12,14,16,18,20]

Prelude> filter even [1..20]
[2,4,6,8,10,12,14,16,18,20]

Prelude> [x | x <- [1..20], even x]
[2,4,6,8,10,12,14,16,18,20]

Note the last expression, which is a list comprehension, very similar to the mathematical notation $\left\{x \; \vert \; \forall\, x \in {1,\ldots,20},\; iseven(x)\right\}$.

Using recursion and list comprehensions we can implement the quicksort sorting algorithm. Given a list, quicksort orders the items as follows:

Choose the first element as a pivot.
Build two lists: the first with elements smaller than the pivot, and the second with elements bigger than the pivot.
Sort these two lists using the same quicksort algorithm.
The result is a list containing the sorted smaller values, followed by the pivot, followed by the sorted list of bigger values.

The Haskell implementation of this algorithm can be very neat:

qsort :: [Int] -> [Int]
qsort [] = []
qsort (x:xs) =
    qsort [y | y <- xs, y <= x] ++ [x] ++ qsort [y | y <- xs, y > x]

Lazy evaluation

Haskell is based entirely on the lazy evaluation, ie, making the calculations as late as possible, and that's when they are needed by someone else. This enables the language to use endless lists, because values of the list are only calculated when another task needs them. For example:

Prelude> take 5 [1 ..]
[1,2,3,4,5]

Prelude> take 10 [1 ..]
[1,2,3,4,5,6,7,8,9,10]

Using this feature, we can implement the sieve of Eratosthenes algorithm, which generates the list of prime numbers.

The sieve of Eratosthenes begins with number 2 (which is prime) and marks all its multiples. It repeatedly takes the smallest number not marked, which is prime because it is no a multiple of any other smaller number, and again marks all of its multiples.

This process returns all primes smaller than a given number: obviously we can not mark all even numbers, because they never end. Moreover, what we can do is to discard all even numbers lower than 1000, for example. Haskell, however, by using lazy evaluation, is able to work with never ending lists and therefore allows a very simple implementation:

primes :: [Int]
primes = sieve [2..]
    where
        sieve (p:xs) = p : sieve [x | x <- xs, x `mod` p > 0]

We save it in the file primes.hs and we can run:

$ ghci
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help

Prelude> :l primes
[1 of 1] Compiling Main             ( primes.hs, interpreted )
Ok, modules loaded: Main.

*Main> take 5 primes
[2,3,5,7,11]

*Main> takeWhile (<50) primes
[2,3,5,7,11,13,17,19,23,29,31,37,41,43,47]

Higher order functions

Higher order functions are functions that take other functions as parameters. An example is the map function, which takes a function as the first parameter and a list as the second parameter, and it applies the function to each item in the list. For example:

Prelude> map (* 2) [1..10]
[2,4,6,8,10,12,14,16,18,20]

Prelude> map (^ 2) [1..10]
[1,4,9,16,25,36,49,64,81,100]

Given a list of integers, and a list of exponents, we can raise each number to the given power:

Prelude> zipWith (^) [1..5] [1..5]
[1,4,27,256,3125]

Here You can find more examples of higher order functions and their use.

Pattern matching

Another interesting feature of Haskell is pattern matching. We can check the pattern (x:y:xs) over a list, so x and y represent the first and second value of the list and xs the remaining elements.

Let us see how we can use pattern matching to implement binary tree traversal in different orders. Let's remember first preorder, inorder and postorder traversals:

* Ways to traverse a binary tree: preorder (left: F, B, A, D, C, E, G, I, H) inorder (center: A, B, C, D, E, F, G, H, I) and postorder (right: A, C, E, D, B, H, I, G, F). The black dots indicate at what time of the route each node is chosen*

And that we can implement it like this:

data Tree a = Node a (Tree a) (Tree a) | Empty deriving (Show)

preOrder :: Tree a -> [a]
preOrder Empty = []
preOrder (Node n t1 t2) = [n] ++ preOrder t1 ++ preOrder t2

postOrder :: Tree a -> [a]
postOrder Empty = []
postOrder (Node n t1 t2) = postOrder t1 ++ postOrder t2 ++ [n]

inOrder :: Tree a -> [a]
inOrder Empty = []
inOrder (Node n t1 t2) = inOrder t1 ++ [n] ++ inOrder t2

The expressions (Node n t1 t2) is where pattern matching takes place.

Input / output

Consider also a small example of input/output: a program that reads the name a person says Hello maca if the name ends in a and Hello maco otherwise (maco and maca mean pretty in Catalan :P):

main :: IO()
main = do
    name <- getLine
    putStrLn $
        "Hola mac" ++
        (if elem (last name) "aA" then "a" else "o") ++ "!"

Product of polynomials with Fast Fourier Transform

Finally, let's see the implementation of polynomial multiplication using the Fast Fourier Transform. The product of polynomials of degree $n$ has a cost of $\mathcal{O}(n^2)$ if we use the trivial algorithm: we multiply each coefficient of the first polynomial by each coefficient of the second polynomial, and then add them as needed, which leads to a quadratic performance. There is a much faster algorithm with a performance of $\mathcal{O}(n\log n)$, which uses the Fast Fourier Transform.

A very good explanation of how this algorithm works and how to implement it can be found here.

Next I show a working implementation, where the FFT is implemented in only 12 lines of code and 10 more lines are used to implement polynomial multiplication.

import Data.Complex

fft :: [Complex Double] -> Complex Double -> [Complex Double]
fft [a] _ = [a]
fft a w = zipWith (+) f_even (zipWith (*) ws1 f_odd) ++
          zipWith (+) f_even (zipWith (*) ws2 f_odd)
    where
        -- Take even and odd coefficients of a(x)
        a_even = map fst $ filter (even . snd) (zip a [0..])
        a_odd  = map fst $ filter (odd  . snd) (zip a [0..])
        -- Compute FFT of a_even(x) and a_odd(x) recursively
        f_even = fft a_even (w^2)
        f_odd  = fft a_odd  (w^2)
        -- Compute 1st and 2nd half of unity roots (ws2 = - ws1)
        ws1 = take (length a `div` 2) (iterate (*w) 1)
        ws2 = map negate ws1

-- Multiplication of polynomials: c(x) = a(x) · b(x)
--     * a(x) and b(x) have degree n (power of 2), c(x) has degree 2n
--   Algorithm:
--     1. Compute f_a = fft(a), f_b = fft(b)  .. O(n·logn)
--     2. Compute product: f_c = f_a · f_c    .. O(n)
--     3. Interpolate c: c = fft^{-1}(f_c)    .. O(n·logn)
mult :: [Double] -> [Double] -> [Double]
mult [] _ = []
mult a b = map realPart c
    where
        n = length a
        w = exp (pi * (0 :+ 1) / fromIntegral n)
        f_a = fft (map (:+ 0) $ a ++ replicate n 0.0) w
        f_b = fft (map (:+ 0) $ b ++ replicate n 0.0) w
        f_c = zipWith (*) f_a f_b
        c = map (/ (fromIntegral $ 2*n)) $ fft f_c (1/w)

To test this program, save it in the file fft.hs, open a Haskell interpreter ghci and load the program: :l fft.hs. We can multiply polynomials with mult [2,1] [3,-2]. Because we are working with floating point numbers, we can see small inaccuracies (eg. instead of 2.0, the system can save 1.9999999999999996). That's why we define round2, a function which rounds numbers to two decimal places. Here are some examples:

$ ghci
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help

Prelude> :l fft.hs
[1 of 1] Compiling Main             ( fft.hs, interpreted )
Ok, modules loaded: Main.

*Main> let round2 f = (fromInteger $ round $ f * 100) / 100

*Main> map round2 $ mult [2,1] [3,-2]
[6.0,-1.0,-2.0,0.0]

*Main> map round2 $ mult [2,1,0,0] [3,-2,0,0]
[6.0,-1.0,-2.0,0.0,0.0,0.0,0.0,0.0]

*Main> map round2 $ mult [3.5,-2,6,1] [3.5,-2,6,1]
[12.25,-14.0,46.0,-17.0,32.0,12.0,1.0,0.0]

*Main> map round2 $ mult [1,2,3,4,5,6,7,8] [0,0,0,0,1,0,0,0]
[0.0,0.0,0.0,0.0,1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,0.0,0.0,0.0,0.0]