Modern JavaScript, TypeScript and You

[thetutlage]

JavaScript community is always on a treadmill. The spec, tooling, frameworks, best practices, in fact everything is on the road to change.

I have been trying very hard to stay away from all the hype for the past few years. And follow the path of simplicity. Web development is not rocket science, and everything is achievable by staying on the simplest path.

Anyways, the main topic of this post is to talk about the state of modern JavaScript and TypeScript.

I will be focusing on Node.js only (this is the place where I spend most of my time).

Imagine JavaScript was launched yesterday

JavaScript has come a long way. It started as a tiny scripting language that turned into a full-blown programming language.

We got introduced to transpilers, bundling, complex toolchains, and whatnot during this time. All that maybe was required in the past to push the language forward and not wait forever for the spec to finalize and then get implemented by different runtimes.

But, using complex toolchains came with a downside. They modified our behavior and now we do not double think about adding Babel, WebPack, Vite, and 100 other build tools to our projects. It has become the second nature for us.

There is a cost to using complex build tools, which is the massive complexity they come with. This is why it is an ever-evolving space because no one is happy with the current style of doing this. Once grunt was incredible, then it was gulp, then wow WebPack, and now Vite.

These tools are great. I am not ranting on them. But I want to look at JavaScript with fresh eyes and find the simplest way to use it every day.

Also, I primarily on Node.js to write backend servers. So, the post is purely focused on that.

So let's see how we can use the greatness of modern JavaScript and some of the surrounding tools like TypeScript and ESLint.

In case you are not aware, node.js support all the following features.

• Native ES modules
• Class private members
• Top-level await
• Import aliases (aka subpath imports)

Getting started

• Install the LTS version of Node (i.e. 16.14.2 as of today).

• Create an empty folder and run npm init --yes inside it. This command will create a package.json file for you.

• Let's open package.json and add a new property type=module inside it. It tells Node.js that you want to use ES modules.

  {
    "name": "esm-ts-and-you",
    "version": "1.0.0",
    
    "type": "module",
    
    "description": "",
    "main": "index.js",
    "keywords": [],
    "author": "",
    "license": "ISC"
  }

• Now, let's create two new files inside the src directory.

  mkdir src
  touch src/user.js
  touch src/greet.js

• Export a dummy user object from the src/user.js file.

  export const user = {
    name: 'virk',
    id: 1
  }

• And import it inside the src/greet.js file to greet the user. Notice that we import the module with the extension. If you omit the extension, the import will break. That's a breaking change when moving from CJS to ESM.

  import { user } from './user.js'
  
  console.log(`Hello "${user.name}". We are using ES modules`)

• Finally, run the code as node src/greet.js.

  ❯ node src/greet.js
  Hello "virk". We are using ES modules

This is the simplest and best possible way to use ES modules. "Breath of fresh air 😌"

Import aliases

Right now, our directory structure is quite simple. But in an actual application, we may have nested folders and import statements across sub-directories.

In the past, you might have used WebPack aliases or AdonisJS aliases, and most recently Vite aliases.

If you are new to aliases, they solve a simple use case.

Instead of importing files as follows

import { user } from '../../modules/user.js'

You can import them as follows (@modules is the alias)

import { user } from '@modules/user.js'

Now, Node.js has built-in support for defining aliases. They are known as subpath imports. You define them within the package.json file.

Note: Subpath imports are not related to ES modules. They work even when you use Common JS.

{
  "imports": {
    "#src/*": "./src/*.js"
  }
}

Once you have defined the subpath import, you can import the same ./user.js file as follows. Notice that we have removed the .js extension from the import statement this time. Because it has been handled by the alias expression ./src/*.js.

import { user } from '#src/user'

console.log(`Hello "${user.name}". We are using ES modules`)

Subpath exports

The consumer of Node packages (distributed as npm packages) are used to import modules by their path. Following is an example directory structure of a package.

.
├── package.json
└── src
    ├── colors.js
    └── prompts.js
├── index.js

• The index.js is the "main" file. You can import it directly using the package name.

  import 'foo-bar-pkg'

• However, if src/colors.js was never exposed from the index.js file. You can still import it by defining the complete path.

  import colors from 'foo-bar-pkg/src/colors.js'

This is good and bad at the same time. Your package consumers can pull in any module they want. However, it makes it harder for you to re-arrange the code internally without breaking others apps.

Usually, it is considered good practice to define the public interface for a package explicitly. Just like classes have public and private properties (yes, JavaScript has them as well, stop making Fun of it 🤪).

With the help of subpath exports you can control which files to expose from a package. For example, if you want to export the index.js file, write the following code line within the package.json file.

{
  "exports": {
    ".": "./index.js"
  }
}

Maybe, you also want to expose the src/colors.js file. But still want to retain the ability to move the file somewhere later during some re-factor. Here's how you can do that.

{
  "exports": {
    ".": "./index.js",
    "./colors": "./src/colors.js"
  }
}

And then import it as follows. Notice that we are not defining the file extension since it is part of the subpath export expression ./src/colors.js.

import colors from 'foo-bar-pkg/colors'

Up until now. I extensively use sub-path imports and exports and do not type extensions manually.

Adding TypeScript to the mix

You just heard about the new language (JavaScript) yesterday, and now someone told you, wait, you can add static types to this language using a new language called TypeScript.

So TypeScript - Types = Valid JavaScript. This is the theory and the goal of TypeScript. Later in this post, we will explore how TypeScript holds to this goal.

Fun fact: There is a dedicated website You might not need TypeScript... (syntax) that tells you that TypeScript is not something you need.

I kind of agree with this sentiment, but not completely. So here's my take on TypeScript.

• I will not use TypeScript to use the modern JavaScript syntax not implemented by Node.js yet. It's better to wait and let the spec finalize. Decorators are a great example of this. Babel and TypeScript implementation are different from the spec. It is in stage 2 right now and has been completely re-written.
• If you can get away without using a build tool, please do. Writing the language as it is, gives another level of joy.
• I use TypeScript mainly for some of its advanced features like Generics, type assertions, and conditional types. Writing all this as a JSDoc comment block is not a great idea. Comments have a history of getting out of sync.

Considering this. I will be making it possible to write AdonisJS apps in pure JavaScript. I will continue to use TypeScript because of the reasons mentioned above in point 3.

However, JavaScript has matured a lot, and we should not force TypeScript on new developers.

---

Alright, a lot of theory. Let's see how well TypeScript plays with modern JS constructs.

• Install TypeScript 4.7, which is in beta right now. This version of TypeScript has decent support for ESM

  npm i typescript@beta

• Create a tsconfig.json file with the following contents. Make sure the module=NodeNext for ESM and TypeScript work great together.

  {
    "compilerOptions": {
      "target": "ES2020",
      "module": "NodeNext",
      "strict": true,
      "outDir": "./build",
      "esModuleInterop": true,
      "skipLibCheck": true,
      "forceConsistentCasingInFileNames": true
    },
    "include": ["./**/*"],
    "exclude": ["./node_modules", "./build"]
  }

• Finally, let's rename all the .js files to .ts. As per our previous example, we will have to rename the following files.

  mv src/greet.js src/greet.ts
  mv src/user.js src/user.ts

• Let's compile the code using TypeScript and see if we get any errors. The compiled code is written within the ./build folder.

  ./node_modules/.bin/tsc

For me, the code compiles fine. However, I cannot run it from the build folder.

node ./build/src/greet.js

# Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/xx/xxxx/src/user.js' imported from '/xx/xxxx/src/greet.js'

The issue is that our subpath import "#src/*": "./src/*.js" is trying to find the src/user.js file within the root of the project and not the build folder.

Suppose we update the subpath import expression to always lookup inside the build directory. Then, our TypeScript code will not compile because, as per the TypeScript compiler, there is no ./build/src/user.js file.

In short, I have not been able to use subpath imports and TypeScript together. The only way to make it work is to use relative paths inside import statements.

Let's open the src/greet.ts file and fix it for now.

import { user } from './user.js'

console.log(`Hello "${user.name}". We are using ES modules`)

Wait, have to notice something???
Even though we are using TypeScript, we will have to import our files with the .js extension. Yes, this is how it has to be.

TypeScript justifies the decision by saying, "Our goal is to make TypeScript - Types = JavaScript" and if they allow .ts extension, they will diverge from their goal.

From the DX point of view, this is a wrong move. I cannot imagine teaching a newcomer to import a file with the .js extension, even though they see the file has a .ts extension in front of their eyes.

But, I still buy the TypeScript argument. Because this goal will help JavaScript in the long run, meaning, TypeScript will not re-invent the ES spec.

TypeScript and JavaScript private members are not the same

Until recently, JavaScript had no concept of public and private class properties. As a result, everything you defined in a class was publicly accessible.

TypeScript, on the other hand, does have private, public, and protected modifiers. The following TypeScript code

class User {
  private firstName = 'Harminder'
  private lastName = 'Virk'

  public fullName = `${this.firstName} ${this.lastName}`
}

Compiles to the following JavaScript code. At runtime, anyone can access the firstName and lastName properties.

class User {
  constructor() {
    this.firstName = 'Harminder';
    this.lastName = 'Virk';
    this.fullName = `${this.firstName} ${this.lastName}`;
  }
}

However, as of today, you can have private class properties in JavaScript. The properties starting with a # are marked as private. This privacy is also maintained during the runtime.

class User {
  #firstName = 'Harminder'
  #lastName = 'Virk'

  public fullName = `${this.#firstName} ${this.#lastName}`
}

The TypeScript private modifier does not create a JavaScript private property.

Remember TypeScript's original goal? TypeScript - Types = JavaScript. So, if they start transforming the private modifier to #, they will be going against their initial goal.

Therefore, even when using TypeScript, you will have to use # and not the private modifier if you want a runtime private property. That kind of makes this modifier redundant in the first place.

Right now, I extensively make use of TypeScript modifiers. However, I will be getting rid of them altogether in my code. Every property will be public by default unless prefixed with a # (which gives both the compile-time and runtime safety)

Running TypeScript without compilation

No one has the patience to re-compile the code every time before they can run it. Also, the TypeScript compiler is not the fastest, so it can become frustrating to wait for 5-10 seconds before you can see the output of your change.

Therefore, many tools are trying to run TypeScript code without the compilation step. They take the TypeScript code and return back JavaScript without performing type-checking.

Like everything else in JavaScript, this space has also become crowded. So I will not talk about all those tools and their differences. Instead, our goal is to pick the most straightforward tool that does the job well in a predictable manner.

Say hello to ts-node

So far, TS node is the best tool that sticks to a single goal of compiling TypeScript to JavaScript within memory. Following are some of the things I like about the Ts node.

• It has a single goal. Not trying to be a universal bundler for browser and Node.
• Uses tsconfig.json rules for compiling the code. Many other tools introduce additional configuration options, making the entire space more confusing and error-prone.
• Has the best support for ESM.
• And now also allows using SWC for all those that need extra speed.

I am not looking for the fastest tool in the market. Instead, I am trying to stay on the most straightforward path that sticks to existing JavaScript and TypeScript standards.

• Let's install the Ts node as follows

  npm i -D ts-node 

• And run your TypeScript code using its ESM loader. The --loader flag uses the Node.js experimental loader hooks to compile the source code on the fly.

  node --loader=ts-node/esm src/greet.ts

• Output. For now, you will have to live with the following warning statement since the loaders API is in the experimental stage.

  # (node:53547) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
  # (Use `node --trace-warnings ...` to show where the warning was created)
  
  Hello "virk". We are using ES modules

Great, but it's not fast

By default, Ts Node performs type checking. I turn it off because I rely on my code editor to do that. Also, I perform type checking when bundling my code for production vs. doing it every time during development.

You can turn off type checking by adding the following property to the tsconfig.json file.

{
  "ts-node": {
    "transpileOnly": true
  }
}

You can run TypeScript code without compiling or type-checking it. Type checking is left on the editor during development, and you must perform it during the production build.

How about even faster?

I have seen people re-writing their apps because the new framework is 30% faster than they are using. Yeah, we all are insane and craving faster, faster, and faster tools.

Usually, tools optimizing for speed comprises somewhere else. Something they usually do not document and something you find after using that tool in the long run.

However, sometimes the speed is achieved by simply using a faster language. This is the case with swc. TypeScript official compiler is written in JavaScript (one huge file), and SWC is written in Rust. No doubt, Rust is a better language for writing compilers.

Let's start by installing the following SWC packages.

npm i -D @swc/core

And adding the following property to the tsconfig.json file.

{
  "ts-node": {
    "transpileOnly": true,
    "swc": true
  }
}

Yeah. It's freaking fast!

SWC caveats

There are a few things that make me a little uncomfortable with SWC.

• Nowhere on their website do they mention being compatible with the official TypeScript compiler. It means they can add/remove features in the future.
• They mention a lot about bundling the frontend code for browsers. I have personally noticed that the tools built to target browsers usually get complicated over time.

The above points can bring some headaches in the future. For example, SWC behaves a little differently from TypeScript on certain features, and then your code will work fine in development. However, it may break in production (if you make the production build using tsc).

I am happy using SWC via Ts-node because if SWC is not something I need in the future, I can just remove one flag and continue using Ts-node with the official TypeScript compiler.

Adding ESLint to the mix

Thankfully, we all have settled on ESLint for linting our code. Feel free to use any ESLint settings you want (as per your taste). However, I recommend following plugins and rules for a better ESM + TypeScript experience.

{
  "env": {
    "es2022": true,
    "node": true
  },
  "parser": "@typescript-eslint/parser",
  "parserOptions": {
    "project": "tsconfig.json",
    "ecmaVersion": "latest",
    "sourceType": "module"
  },
  "plugins": [
    "unicorn"
  ],
  "rules": {
    "unicorn/prefer-module": "error",
    "unicorn/prefer-node-protocol": "error"
  }
}

• ESLint is a JavaScript linter. Therefore we need to use a specialized parser ‌@typescript-eslint/parser for TypeScript projects.
• Next, I use ESLint plugin unicorn to enforce a couple of rules.
  • unicorn/prefer-module disallows globals available in CJS only (like __dirname). It also disallows require, exports and module.exports.
  • unicorn/prefer-node-protocol enforces you to use the node: prefix for native modules. For example import { readFile } from 'node:fs'

Conclusion

Not sure about you. But I find the JavaScript ecosystem very confusing. We are adding layers over layers over layers.

I know this culture will not change sooner. However, JavaScript as a language has matured a lot, and Node.js implements all of the ES features.

Except for TypeScript, I want to keep my development toolchain simple and easy to work with.

Footnotes

• Ts Node transpileOnly example does not work right now. I have started a discussion on the same in the Ts Node repo.

[Zizaco]

Thank you for this writing. The node/js ecosystem is indeed way too complicated.

I will save this step-by-step to recommend for people just getting into node.

Btw, have you already checked Deno? It tackles the key issues that you described. I recommend it if you have the time; I'm sure you will like it (no transpiler, no bundler, no bikeshedding, no npm, no yarn, a sane binary that does everything you expect it to).

The Deno ecosystem is not as mature, but I wish it was (or at least that it had an AdonisJS-like framework) because IMO it makes the js/ts experience butter smooth. As smooth as if it was rust or like if node "was launched yesterday."

[thetutlage]

Yup, deno has made some good decisions upfront. But just imagine, if I even consider moving AdonisJS to deno, it will be extremely difficult for me.

Also, I am trying to focus more on user land features vs internal technical features. For example: Building a queuing system for Node, or having a SAAS boilerplate for AdonisJS.

[Zizaco]

I understand. I didn't mean that you should move Adonis to Deno. And I totally agree that it's better to focus on User/DX features than on the technical features. Also, AdonisJS already abstracts much of the complexity you just mentioned in the post 🙂.

Nevertheless, I keep my fingers crossed that one day Deno will have a richer ecosystem (possibly to be the runtime of Adonis v10 😉) or that the Node ecosystem will become more lucid.

Microsoft's type syntax proposal may help make the use of Typescript easier in Node.

[thetutlage]

Microsoft's type syntax proposal may help make the use of Typescript easier in Node.

Yes. Really looking forward to this