Reduce typescript package size

Search Terms

size, bloat, install

Suggestion

The typescript package is large, and it only getting larger.

Version 3.1.3 is a whopping 40MB.

Use Cases

TypeScript is used in many contexts.

A TypeScript formatter (e.g. prettier) does not need an entire compiler. It only needs a parser. And 45MB scripted parser is orders of magnitude larger than one would normally expect. (For reference, the installed npm package for Esprima -- the most compatible and compliant ES parser in the ecosystem -- is a mere 0.3MB.)

Examples

Solution 1: Split up packages

typescript (existing package; depends on typescript-compiler, typescript-parser, typescript-server)
typescript-compiler (depends on typescript-parser)
typescript-parser
typescript-server (depends on typescript-compiler)

Optionally, there could be separate packages for typescript-config and typescript-i18n.

Solution 2: Don't duplicate code

There is a lot of code duplication between

lib/typescriptServices.js
lib/typescriptServices.js
lib/tsserver.js
lib/tsserverlibrary.js

Don't duplicate the code.

Checklist

My suggestion meets these guidelines:

This wouldn't be a breaking change in existing TypeScript / JavaScript code
This wouldn't change the runtime behavior of existing JavaScript code
This could be implemented without emitting different JS based on the types of the expressions
This isn't a runtime feature (e.g. new expression-level syntax)

Some backstory in #23339.

Interesting reading, thanks.

TypeScript [2.9.0] has doubled in size since v2.0.0 - now 35 MB

It was "fixed" by #25901, released in 3.1.1, which was 40MB. 🙁

It won't be hard at all to shrink the package size. For example, lib/tsserver.js and lib/tsserverlibrary.js are 98% identical.

$ du -b node_modules/typescript/lib/tsserver.js node_modules/typescript/lib/tsserverlibrary.js
7290127	node_modules/typescript/lib/tsserver.js
7308140	node_modules/typescript/lib/tsserverlibrary.js
$ comm -12 <(sort node_modules/typescript/lib/tsserver.js) <(sort node_modules/typescript/lib/tsserverlibrary.js) | wc -c
7207205

And 99% of lib/typescript.js is identical to those.

$ du -b node_modules/typescript/lib/typescript.js
6859801	node_modules/typescript/lib/typescript.js
$ comm -12 <(sort node_modules/typescript/lib/tsserver.js) <(sort node_modules/typescript/lib/typescript.js) | wc -c
6850490

And lib/typescriptServices.js is byte-for-byte identical to that.

$ sha1sum node_modules/typescript/lib/typescript.js node_modules/typescript/lib/typescriptServices.js
0cff9734eba3d721a7ba3c72026e16f267610e24  node_modules/typescript/lib/typescript.js
0cff9734eba3d721a7ba3c72026e16f267610e24  node_modules/typescript/lib/typescriptServices.js

And 99% of lib/typingsInstaller.js is identical to that.

$ du -b node_modules/typescript/lib/typingsInstaller.js
5285788	node_modules/typescript/lib/typingsInstaller.js
$ comm -12 <(sort node_modules/typescript/lib/typingsInstaller.js) <(sort node_modules/typescript/lib/typescriptServices.js) | wc -c
5246999

And 80% of lib/tsc.js is identical to that

$ du -b node_modules/typescript/lib/tsc.js
3912404	node_modules/typescript/lib/tsc.js
$ comm -12 <(sort node_modules/typescript/lib/typingsInstaller.js) <(sort node_modules/typescript/lib/tsc.js) | wc -c
3219205

That's nearly 30MB of duplication just in those few files (and this doesn't even include declaration files).

I can't begin to guess at the kinds of design decisions that produce this (or what kind of compatibilities the TS team needs to support), but I trust there is a solution the maintainers would be happy with.

I can't begin to guess at the kinds of design decisions that produce this (or what kind of compatibilities the TS team needs to support), but I trust there is a solution the maintainers would be happy with.

It's done this way so that every file can be used by itself without having to deal with the nastiness of modules in JavaScript. Every file is a functional library/program in itself. I think that is a great thing, at the cost of some disk space.

Reading the linked issue #23339, it appears that it desire is in fact to (eventually) use modules.

#23339 (comment)

If we used modules, we'd be able to share each file and avoid this duplication

it is something we want to do, but no plans for the short term. that is where the majority of savings would come from.

nastiness of modules in JavaScript

ES module systems in general can be hit-and-miss, but reminder that we're talking specifically about an npm package.

npm, npm packages, node_modules, package.json, etc. are relate to Node.js (or clones) which supports CommonJS. Right?

I have two ideas, but I am not sure which one is better.

split code in the source code
for now, some common utils or helper has been shared with a different component, we could split them by function, eg: utils.ts -> utils.ts( common), utils.factory.ts(depend on factory), utils.emitter.ts(depend on emitter), etc.
if you want a factory or emitter only. just create a tsconfig.json file that include the depended file,
analyze and transform the bundled file
the namespace has been compiled to many iife and injected the namespace instance,
we could compile with target esnext and merge those iife, then transform the ts.xxx = xxx to export xxx,
and then, we could pack them as a normal esm project and tree shark

Daniel Rosenwasser (@DanielRosenwasser)

ping Daniel Rosenwasser (@DanielRosenwasser)
What do you think about that🧐?

I am skeptical that tree-shaking is useful for shipping our own package because presumably everything we ship is used in some capacity, or is part of our public API - at which point, our consumers would actually be the ones winning from tree-shaking.

Splitting source on its own can help, but practically speaking the larger components like services and TSServer will need the entire core compiler.

I think that converting to modules is the most practical and obvious way to avoid duplicating most of the contents of tsc.js 3+ times.

A simpler solution: inspired from Busybox.

Combine N near-duplicate files into 1 polymorphic file that can do N things based on a parameter passed in.

It would introduce a performance overhead of parsing tiny % of unnecessary JS code, but can make the tool integration story way simpler. Maybe worth it?

One trivial way to know which feature is expected would be to directly copy Busybox approach: symlink all the duplicate files and differentiate at runtime based on the __filename. Saves disk space, package size, bandwidth. There are more interesting options too.

Ryan Cavanaugh (@RyanCavanaugh)

From speaking with Ryan Cavanaugh (@RyanCavanaugh), it sounded like Orta Therox (@orta) was interested in working on this.

+1 for splitting up typescript into multiple packages. One major benefit would be that these individual packages (other than the "typescript" package) could use semantic versioning on at least their APIs then other libraries could just depend on the packages they need. Right now it's kind of a pain to maintain a library that has a peer dependency on the typescript package (without being super strict about the supported version).

This matters when opening a repo on an online IDE where there is no cache. My home connection is ~2MB/s, so even in tarball TS still adds few seconds when I open a Stackblitz repro for Vite.

every modern package manager uses hardlinks to a global cache

Neither npm nor yarn use a global cache. (Unless Yarn is PnP mode, which brings a number of issues.)

overall most people only download each version of TypeScript once

There are over 2,800 versions of TypeScript. The chance that two different projects happen to install the same exact version is very low.

Even for a single npm install which dedups as much as possible, right now I'm looking at a project with 5 TypeScript versions. (Why? jsii, postcss-loader, prettier-plugin-organize-imports, puppeteer-core, cosmiconfig-typescript-loader, plus the version for the project itself.)

This matters when opening a repo on an online IDE where there is no cache. My home connection is ~2MB/s, so even in tarball TS still adds few seconds when I open a Stackblitz repro for Vite.

That's certainly true. It's a shame that these systems do not cache their artifacts.

Neither npm nor yarn use a global cache. (Unless Yarn is PnP mode, which brings a number of issues.)

Yarn 3 supports hard linking (https://yarnpkg.com/configuration/yarnrc#nmMode). If you're still using Yarn v1, you're not going to get any new features at all.

I was wrong about npm; it has a global cache but it copies the files.

There are 2,800+ versions of TypeScript. The chance that two different projects happen to install the same exact version is very low.

Even for a single npm install which dedups as much as possible, right now I'm looking at a project with 5 versions. (Why? jsii, postcss-loader, prettier-plugin-organize-imports, puppeteer-core, cosmiconfig-typescript-loader, plus the version for the project itself.)

There should really only be one TS version in a project; if this is happening, then some package is over-restricting what version of TS it needs. All modern package managers allow you to override versions within a workspace, and I would think it'd be safe to do that if space is a concern and your package manager can't hardlink.

It's also misleading to say that there are 2,800 versions of TypeScript; there are only a handful of stable releases. The rest are nightly builds.

People shouldn’t have to override Typescript versions. The project I’m working on now has 70 dependencies and if they all required post-install customization npm would be pretty unusable.

People shouldn’t have to override Typescript versions. The project I’m working on now has 70 dependencies and if they all required post-install customization npm would be pretty unusable.

I'm referring specifically to doing this in npm:

"overrides": {
    "typescript@*": "$typescript"
},

Or in yarn:

"resolutions": {
    "typescript@*": "$typescript"
},

Or in pnpm:

"pnpm": {
    "overrides": {
        "typescript@*": "$typescript"
    },
}

I am not referring to any sort of post-install patching, but just asking the package manager to resolve to a single version.

The point is that an override only seems reasonable because other dependencies don’t require any extra setup. NPM repos are supposed to be low-effort installs and typescript should be no exception.

It's a shame that these systems do not cache their artifacts.

There should really only be one TS version in a project

npm; it has a global cache but it copies the files.

Yes, as you say, IDEs, package maintainers, and package managers should be aggressively deduplicating redundancies.

....

And TypeScript should be doing the same. (Right now it's something crazy like ~75% duplicate code.)

Yes, again, #27891 (comment) removed one more copy, and #51440 will remove even more (down to the absolute minimum of 2 copies one can have when shipping both CJS and ESM). I'm not sure what else I can say, I was just originally attempting to explain that a large bulk of situations do not benefit from the effort to lower the package size.

I think it'd be useful for people to be a bit more specific about what they care about so we can tailor our efforts.

For time-over-wire, deduplication isn't a great savings, since each additional copy is a tiny increment (compressed checker.ts (2 MB) is 416k, compressed 4x checker.ts is 418k)

For space-on-disk, uh, I'm going to need some more details. It's not 1998 anymore. 40 MB is 0.04% of a terabyte. If the problem is that there are 35 copies of TS due to how a package manager behaves, going from 35 to even 10 is going to be a much bigger than anything we could plausibly do. That's a package manager problem, not a TypeScript problem, it's unrealistic to expect projects to put their effort into slimming down instead of having package managers duplicate less.

For bundling into other projects like web IDEs, treeshaking is going to be a big part of any successful strategy here. Identifying places where we can be more shakeable is a good thing.

For space-on-disk, uh, I'm going to need some more details.

I see where you're coming from, but npm is still the most commonly used package manager. The problem is npm, definitely, but who knows when this will change? Anything that reduces TS size definitely has impact on the ecosystem.

Same is true for the "time-over-wire" thing, shipping 1 MB less is probably not doing anything for one person, but if you multiply 1000 kb by 43 million downloads weekly, the picture looks totally different again.

As far as I can tell, the details you need are written down here: https://github.com/pi0/tslite#how

There are plenty of cases where cache isn't available and the size of the package matters – for over the network size, number of files that need to be written, and amount of JS that needs to be parsed at execution time.

Container builds. Installs in containers typically have no cache (would require volume mounting, etc), so installs are slower. The resulting container size also matters for a number of reasons, from execution time, to registry push time over slow networks, etc, etc. Ppl typically try to keep their container sizes to a minimum, so small packages help here.
CI builds. Similar to above. Often done in containers. Many CI systems have caching abilities, but they can be complicated to setup – and many don't. So often typescript is being installed from scratch each time, for every single build, just adding time to every single build.
Serverless environments. Environments like Lambda, Google Cloud Run, Cloudflare Workers, etc execute much better with smaller zipfiles/container sizes. Reducing dependencies in these environments is a known best practice. Large packages are frowned on. Some have limits on size.
Performance. The more files parsed, the more JS parsed, the slower a package is to start.

If the problem is that there are 35 copies of TS due to how a package manager behaves, going from 35 to even 10 is going to be a much bigger than anything we could plausibly do

35 to 10 is a 71% reduction.

The latest version (5.2.2) has tsc, tsserver, tsserverlibrary, and typescript which total 32MB but have only 9MB of unique content.

Removing the duplicate code drops the package from 41MB to 18MB, a 56% reduction.

So.....actually, there is a lot that TS can do.

it's unrealistic to expect projects to put their effort into slimming down instead of having package managers duplicate less

The problem is largely a synthetic one introduced by TS's bundling. The source code (excluding tests) is only 32MB.

Ryan Cavanaugh (@RyanCavanaugh)

there are 35 copies of TS due to how a package manager behaves, going from 35 to even 10 is going to be a much bigger than anything we could plausibly do

Tangential, but if you want to go that route Ryan Cavanaugh (@RyanCavanaugh) , there's a four-year PR open for PnP to dedup installs, maybe it could be get some eyeballs :)

#35206

Totally agree! I wrote a small utility to extract the parameters of a function, and leveraged the TS compiler library for obvious reasons.

Works beautifully... but I have to include Typescript's 8 MB dependency, which effectively renders it useless.

The Typescript library needs to be tree-shakable! More to the point... why isn't it ALREADY tree-shakable??

My use-case is a web IDE

I want to include a typescript playground for a library I'm making so people can try it out easily. Got here trying to figure out how to lower the typescript dependency size.

Navigation Menu

Reduce typescript package size #27891

Search Terms

Suggestion

Use Cases

Examples

Solution 1: Split up packages

Solution 2: Don't duplicate code

Checklist

46 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Reduce typescript package size #27891

Description

Search Terms

Suggestion

Use Cases

Examples

Solution 1: Split up packages

Solution 2: Don't duplicate code

Checklist

Activity

mattmccutchen commented on Oct 13, 2018

pauldraper commented on Oct 14, 2018

MartinJohns commented on Oct 15, 2018

pauldraper commented on Oct 15, 2018

Kingwl commented on Jan 18, 2019

Kingwl commented on Jan 18, 2019

DanielRosenwasser commented on Jan 18, 2019

mihailik commented on Apr 23, 2019

DanielRosenwasser commented on Sep 10, 2019

dsherret commented on Sep 11, 2019

46 remaining items

ArnaudBarre commented on Oct 6, 2023

pauldraper commented on Oct 6, 2023

jakebailey commented on Oct 6, 2023

spacecowgoesmoo commented on Oct 6, 2023

jakebailey commented on Oct 6, 2023

spacecowgoesmoo commented on Oct 6, 2023

pauldraper commented on Oct 6, 2023

jakebailey commented on Oct 6, 2023

RyanCavanaugh commented on Oct 6, 2023

kurtextrem commented on Oct 7, 2023

mhart commented on Oct 7, 2023

pauldraper commented on Oct 8, 2023

pauldraper commented on Oct 8, 2023

karmaniverous commented on Aug 22, 2024

olsonpm commented on Dec 22, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions