The unreasonable effectiveness of the Julia programming language

No party like a virtual conference party in programming language

I have met many happy and excited scientists lately. Of course, “meeting” in a virtual sense, since conferences and other opportunities to meet scientists in the meat room are as good as impossible. Most scientists believe in the germ theory of disease.

However, these scientists and mathematicians are excited about a new tool. It is neither a new particle accelerator nor a supercomputer. Instead, this exciting new tool for scientific research is … a computer language.

How can a computer language be exciting? Certainly some are better than others depending on your purposes and priorities. Some run faster while others are faster and easier to develop. Some have a larger ecosystem, so you can borrow battle-hardened code from a library and do less work yourself. Some are good for specific types of problems while others are good for general purposes.

For scientists who do math, languages, the quality of compilers and libraries and of course the machines they run on have always been important. For those whose job it is to simulate the atmosphere or design nuclear weapons, Fortran has been the traditional tool of choice (and often is, although there is now more competition). This language has dominated the market as compilers are available that the largest supercomputers can use. For the current generation of data scientists, Python is currently popular because of the dynamism of its ecosystem, interactivity, and fast development cycle.

Six years ago I wrote on these pages about the continued importance of Fortran in scientific computing and compared it to several other languages. I ended this article predicting that in 10 years' time a new language called Julia had a good chance of becoming the one scientists would turn to when dealing with large numerical problems. However, my prediction was not very accurate.

Julia actually only needed about half of this time.

Enough excitement for a con

Talking to scientists in recent years, the Julia computer language has really sparked a new wave of excitement in the industry. But when I wrote about its potential, I didn't understand why the language would take off.

I based my assessment on Julia's unique combination of practical syntax and uncompromising performance. At the time, even though Julia was before 1.0, there was already a lot of excited chatter. Julia appeared to have solved the "bilingual problem" – a conundrum that Python programmers and users of other expressive, interpreted languages ​​often face. You write a program to solve a problem in Python and enjoy its convenient syntax and interactivity. The program works with a trial version of your problem. However, if you try to scale it to something more realistic, it will be too slow. It `s not my fault. Python is inherently slow – something that may not be important to some types of applications but will matter to your large-scale simulation. Eventually, after using various techniques to speed this up but only making modest gains, you will need to rewrite the most time consuming parts of the calculation in C (most common). Now it's fast enough, but now you also have to manage code in both languages, hence the bilingual problem.

Although Julia's solution to this problem has made scholars and others interested in the language, this is not the reason for the newfound excitement surrounding the platform. There's something else.

Enlarge /. The banner for JuliaCon 2020

While I was working on this article, this year's JuliaCon, the annual Julia conference, was taking place (online of course). Usually, the schedule for a computer meeting is filled with titles on subjects such as programming, compilers, algorithms, optimization, and other computer science subjects. And while there was plenty of it at this year's Julia Gathering, flipping through the titles leaves the impression that you've stumbled upon a science conference. There are presentations on all topics from fluid dynamics to brain imaging and language processing. Despite the amazing variety of areas, viewing the presentations gives a sense of community about a common stance that appears to have been influenced by the free software movement.

Every code is on GitHub. If you are interested in using someone else's algorithm in your research, you can read the source and have access to the latest version as it is developed. Scientists of a certain age will know how different this is from previous computer research. In the past, code rarely left the lab.

The Julia community is also united by something else: a shared joy in the magical power (this word appeared more than once) of Julia to make code collaboration and reuse easier. Consider just some of the hymns of praise from the JuliaCon 2020 moderators:

This is one of the things that make Julia so powerful in solving these problems (…). This integration gives Julia an advantage over other languages ​​(…). We were able to develop these solutions in a very short time:

León Alday, molecular modeling

Julia is really the language that makes such a project possible:

George Datseris, Dr. Watson, a research assistant

It is a pleasure to program Julia:

Mauro Werder, thickness of glacier ice

The Julia language (…) is a particularly agile tool:

Valeri Vasquez, Disease Vector Dynamics

Julia was the obvious choice:

Rafael Schouten, room simulations

(Julia allows) I can use tools from different disciplines to advance cancer research:

Meghan Ferrall-Fairbanks, Tumor Dynamics

This work was in Julia because of the beautiful abstractions that allow very general code to make very beautiful:

Vilim Štih, Zebrafish Brain Dynamics

It's really nice to have a quick language that you can write anything with. (…) But what really impresses me these days is something else – Julia can somehow increase my productivity (…). Julia makes it easy to think on the right level of abstraction. ":

Petr Krysl, Partial Differential Equations

Why it is great to do science in Julia (…) Interaction between packages = pure magic !:

George Datseri's analysis of music performance

These scientists have all found that Julia improves collaboration opportunities, making it easier than ever to include the work of others and allow them to write code that others can use in unforeseen ways. The key to these forces lies in Julia's solution to another old puzzle, this time from computer science – the problem of expression.