Coderek's blog

Loneliness is the gift of life

Modern JavaScript loading techniques

Modern JavaScript projects are distinct from traditional ones. The use of NPM system for client side projects enables JavaScript developers to easily use shared libraries at the expense of exploded size of the project. The complexity increases significantly as well. Old school software development methodologies are needed in JavaScripts project.

The old way to load JavaScripts in webpages is to use <script> tag. However, as JavaScript has the potential to modify the page, the browser usually blocks the rendering of subsequent HTML until the prior scripts are loaded and executed. This results in the page UIs being rendered with inconsistent delays. This delay is more visible with large amount of scripts.

There are several solutions to this.

  1. Move script tags to the end of the body tag
  2. Add defer or async attribute to the script tag so it loads asynchronously
  3. Dynamically insert <script> tag to load JavaScript

These solutions solves the problem to a certain degree, but none of them fix the problems entirely. For example, solution 1 will not change overall page loading time. Solution 2 doesn't work in all browsers. Both 2 and 3 doesn't have control on the execution of the loaded scripts.

We need to have a proper loading system to load large size JavaScript projects. The system must satisfy the following conditions:

  1. Lazy loading; Load scripts only when needed
  2. Dependency management; Load dependencies recursively or load the bundles that contains all dependencies smartly
  3. Load asynchronously; Do not block other content or script in the page
  4. Works in all browsers

These requirements are crucial for the performance of large browser-based JavaScript applications. There are many advanced solutions like Webpack and SystemJS. In this article, I want to discuss how these tools solve the problem.

Let's take SystemJS as an example.

The essential way that SystemJS loads scripts is through Ajax or XMLHttpRequest object. It wraps the request with a promise. And when the promise is fulfilled, it evaluates the script at the client side.

With promise it can load and execute the scripts sequentially without blocking. It can also load multiple scripts concurrently and know when the loading completes.

The benefit of using Ajax to load the scripts is that the scripts can be loaded as plain text and the client has the freedom to interpret the content.

For example, you can load ES modules in a non compliant browser using SystemJS then transpile it using babel to es5 syntax. (pre-requisite is to have babel standalone loaded in browser)

Babel has SystemJS plugin which transform all the import...from... statement to system.register(['dep1', 'dep2', ...], function ... . This is essentially the AMD format. When used, it registers modules and dependencies in the SystemJS registry.

When loading a module with dependencies, it look up the path of each dependencies in the registry, if not found, it will try to resolve it to a url and load from that. This is recursively done. One way to optimize it is to use depCache config variable to specify a flat array of dependencies.

Loading from the url is good since you can point it directly to node_modules folder. The downside is that the loaded module can only require/import modules that is recognizable by SystemJS.

Since the module to load doesn't have to be a JavaScript file, it can be anything. For instance, should the loaded text be HTML, it will be transformed into a JavaScript function that returns a HTML text. It can also be CSS file, when transformed, it will become a function that applies style to the entire page. Of course you'll need a loader for each of these.

If you do not want to transform things in the browser, you can of course transform them at server side, and just load plain es5 JavaScript in the browser. SystemJS production build is optimized for this as it doesn't contain the plugin system which is not used for static build.

Compiling code at server side is entirely a separate process. It can be done using a task runner like gulp or grunt. However when it's done it needs to hook back to SystemJS. For example, when bundling code, SystemJS needs to know what modules in which bundle. This is done in the SystemJS configuration.

I would suggest using SystemJS as a bundler for prototyping. The configuration can be really problematic for large projects as I can imagine.

Webpack is a much easier and more powerful alternative. Webpack does everything offline. It has pretty good defaults. It doesn't support loading URL as it overcomplicates things. The dependency loading is smooth and easy. The output of a built Webpack project is just JavsScripts. It's easy to include the files into a web page of our choice. The compilation pipeline is pluggable. You can plug in babel to transpile scripts depending on the extensions. It has built in support for tree shaking. For code using ES module syntax the final built size will be optimized.

Webpack also support dynamic import syntax. It smartly detect dynamic imports and create separate bundles for each one, so we can lazy load them.

I think Webpack solves the problems at the beginning of this article pretty well. It is a blessing for modern JavaScript developers. Every serious JavaScript developer should know it well.

(End of article)