Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I have a lerna project which contains two identical packages (named p1 and p2)

Both p1 and p2 include a 3rd party package – for this test I’ve used eosjs@beta, which is about 50KB

If I then create an example react project and include P1, the package size grows by 50KB as expected, but what’s surprising me is that when I add p2 … it grows by another 50KB.

One would think that because p1 and p2 are using the same 3rd party library, that one reference to that library would be compiled into the example project. But that’s not what seems to happen.

Example repo here: https://github.com/warrick-eosny/sizetest

The growth of the package looks as follows:

ls examples/sizetest/build/static/js/ -lah

Before I reference p1

    117K 1.13eeb203.chunk.js     
    1.4K main.2170ea98.chunk.js  
    1.5K runtime~main.229c360f.js  
After referencing p1 
    165K 1.75baab58.chunk.js  
    3.7K main.36960386.chunk.js  
    1.5K runtime~main.229c360f.js  
After referencing p1 and p2
    212K 1.57bb37cb.chunk.js  
    6.4K main.491260eb.chunk.js  
    1.5K runtime~main.229c360f.js  
The project in the examples folder was created using:  
npx create-react-app sizetest –typescript  
Both the p1 and p2 packages were created using:  
yo node-typescript-webpack  
Why is does the example build keep growing .. surly webpack is smart enough to only include one reference. 
=============== UPDATE ==================  
The "removing duplicate code" section here seems like it should solve my problem:
https://developers.google.com/web/fundamentals/performance/optimizing-javascript/code-splitting/#spitting_code_by_multiple_entry_points  
But that doesn't seem to do so. 
I ran "yarn eject" in the project folder and then added the suggested config: https://github.com/warrick-eosny/sizetest/blob/master/examples/sizetest/config/webpack.config.js#L196-L211   
Removed the uglyfiy section so that the output is readable  
Run the build again   
This does produce a commons file but when you look at the content:
grep \@module build/static/js/commons.bd2f73cb.chunk.js
you can see that the code is still being duplicated  
 * @module Serialize
 * @module Numeric
 * @module RPC-Error
 * @module Serialize
 * @module Numeric
 * @module RPC-Error
 * @module API
 * @module JSON-RPC
 * @module API
 * @module JSON-RPC
Neither a monorepo as a concept nor Lerna as a tool are meant to do such kind of implicit "improvements". This may have unwanted side effects (for example if P1 and P2 depend on different versions of eosjs or where each package initiates an own instance of some package class).
Another reason speaking against doing what you are expecting here, is that packages in a monorepo still can be deployed independently from each other because they don't rely on the same reference of a package.
Utilizing a monorepo is as far as I know the only way to achieve what you are looking for. However, the monorepo just manages your codebase in one place. If you want to use the same reference of eosjs in both your packages, move it up into the root level package.json, but then you will also have to deal with a bunch of other problems that you might not expect yet. You can do it manually for your self-maintained monorepo packages or by hoisting  for external packages with Lerna: https://github.com/lerna/lerna/blob/master/doc/hoist.md
Yarn Workspaces is what Lerna uses under the hood to achieve hoisting and might also help for understanding.
Webpack doesn't know about being in a monorepo unless you told it somehow, too. Its working independently of Lerna or monorepos.
                Sure, but let's remove the fact that it's a monorepo using lerna and say I was just including two npm packages that use the same package and version. It would seem like pretty good optimization to be able to remove the duplication.
– Warrick FitzGerald
                Feb 9, 2019 at 16:19
                @WarrickFitzGerald The linked resource to Webpack doesn't cover your case, it's meant to prevent code duplication inside a single package when you utilise code splitting (for example if you want to lazy load parts of a site only when named parts are requested by the user instead of loading the whole chunk at once. Then you can prevent Webpack to bundle a common package to be packed inside each of the smaller chunks). Your case is different. It would still work in a monorepo and with Lerna by moving the common packages up to the root level package.json as I already said in my answer.
– Maxim Zubarev
                Feb 10, 2019 at 0:41
                Other than that, if you remove the fact of a monorepo or lerna, there is no way of using a single reference of a common node package (how should webpack know which reference to make the "master" reference, or should it hoist by itself?). If an external node packageA has another packageB as a dependency, you should tell the maintainer of packageA that it might be a good idea to move packageB to peerDependencies so you can include it in your main project package.json and it will be resolved once.
– Maxim Zubarev
                Feb 10, 2019 at 0:47
                Thank you, after much experimentation I was able to confirm your thoughts on this. I created a simpler example in case anyone else wants a simple test bed. github.com/warrick-eosny/sizetest-hoist
– Warrick FitzGerald
                Feb 11, 2019 at 12:46
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.