There is no doubt that when it comes to technology preferences, the software development industry can sometimes be likened to a bunch of headless chickens. Developers will often jump from one tech to the next, waxing lyrical about the virtues of whatever stack tickles their fancy at that given moment.
For whichever reason, this statement is most true when it comes to JavaScript development.
So when many “mature” (in years i.e. anyone over the age of 30) developers hear about Node.js their heckles can get up. The first impression is that all these youngsters (yes, JavaScript is a young language, attracting young developers) shouting about how great Node is have most likely never deployed an enterprise grade information system to production.
Having used more traditional backend languages as well as Node.js in production, I would argue that 90% of the time, when it comes to building information systems, Node.js is not the right choice. The primary reason is that for 90% of systems, you want the backend to be synchronous i.e. have your code execute sequentially. This is all the more important when dealing with relational databases.
Having said that, my main gripe with Node has nothing to do with its asynchronicity (a very powerful thing in fact). It’s not actually a gripe with Node itself, but with how many of its developers tend to architect systems.
Very often, instead of carefully designing a robust data model, data is just shoved, serialized and duplicated into “NoSQL” document stores such as Mongo and the like. The approach is top down, instead of bottom up. First comes the language (Node) and how it likes to store data (as arrays or dictionaries) and then you bubble that down and store it as such (as a mess of dictionaries and arrays). The traditional approach is bottom up. Design a robust data model first and then worry about which language / framework will maintain that data.
Having developed software for 15 years, I can without a shadow of a doubt say that the most critical success factor of an information system is to have a well designed, relational, normalised database.
Dealing with relational data is just not a strength of Nodes due to its asynchronous nature. With a relational DB you often would need for one query to finish and provide results, which then get used for the next query and so forth. For 90% of systems, where one is dealing with moderate load and moderate data, using the traditional methods of software development are still best. This means a proper, well designed database on SQL Server, Postgres or MySQL and a synchronous (ideally strongly typed) programming language on the backend.
However, there are the 10% of systems where Node is an outstanding fit (and I’m not referring to the obvious such as chat systems. Plenty of blog posts telling you how great Node is for building a chat system. Who the hell wants to build a chat system? I digress...).
Let me also just add a caveat right here that any information system we build will still be on a relational database. Yes, we will shard and denormalize where needed but don’t ask us to put data into Mongo. And on another side note, Node works wonderfully well with Postgres and the excellent pg-promise library, making synchronous programming through async / await a breeze.
Back to the 10% of systems where Node is a good (no, excellent) fit.
We are currently involved in building a survey system, generating graphs based on vast volumes of data. The system parses large CSV files and inserts their data into Postgres before it’s reported on via Highchart graphs. The system is built on a Vue.js single page app on the front end, with a Node.js API.
Node has shone. Had we attempted this in PHP or Python (using both in their traditional, synchronous forms, as one typically does) I fear we would have had to pay the client their money back.
The following practical examples have shown us the power of Node.
1) We can make a specific choice on both the front-end and backend which API calls to run synchronously and which asynchronously. We have chopped and changed until we found the right balance of async vs sync, i.e balancing speed of execution with load on the database. JavaScript makes this easy. With Python or PHP it would have been difficult. These languages want you to make everything synchronous. When you have a page, generating 30 charts, with multiple lines over 6 months, with each point on the graph hitting a table with over 100 million records (to put it simplistically), having this done in one blocking operation would most likely have resulted in a timeout and out of memory error as well as poor user experience. It would also have made developing and debugging a nightmare.
2) There was a requirement to parse around 50 CSV files, each more than 10 MB. With Python/PHP, this would typically be developed to run in one blocking operation. Get all CSV files, loop through each cell and handle. What happens? Timeouts and out of memory errors before it even gets past the 5th CSV, with the developer having no visibility on errors or progress. With Node, it just handles it one row at a time, printing to the console for visibility and handling each rows as it goes; and it did so incredibly quickly. We could have thrown an infinite amount of CSVs at Node and it would not have timed out or run out of memory. It would have just chugged along, eating the elephant one bite at a time, until done.
Learnings from all of this is that you need to choose wisely, know the pros and cons that apply to real world systems of the various tech stacks and not get caught up in hype (or even anti-hype).
Most likely, the system you need will be more maintainable and stable using a Python/PHP/.NET based MVC framework; most:
- Come with a lot of power out of the box (batteries included)
- Enforce best practice development conventions
- Have security built in
- Enforce sane application architecture/structures
Conversely, it’s much easier to make a mess of Node applications. No structure is enforced and the quality of code largely depends on the capabilities of the individual setting up the project. This is a dangerous thing.
However, as per the examples above, there are cases where Node not only shines and becomes the right tool for the job.
You may very well be one of the few that would greatly benefit from a Node.js backend, you just need to know whether you fall into this camp; and that’s the tricky bit.