v1.0.0 #52

melck · 2019-12-15T11:30:15Z

Hello,

Now that we have tests. I want to rewrite plugin to es6 syntax and add following changes :

Support only webpack >4
Handle splitChunk option of Webpack to add files related of a chunk.
Add integrity option to handle https://www.w3.org/TR/SRI/

What do you think ? I'll start a pull request to show you changes that i want to made

The text was updated successfully, but these errors were encountered:

owais · 2020-01-08T13:12:45Z

Sounds good but since we decided to only support webpack>4, we have a good opportunity to improve other things and break backward compatibility.

One thing I'd like to do is to default to relative paths and remove the relative path option. We should also remove the path key from stats file and instead add relativePath and absolutePath keys. This will ensure other tools that consume the stats file break in an obvious way.

owais · 2020-01-08T13:18:21Z

Another thing I'd like to do is to review the output of stats file and try to make it more generic. I think the stats file shouldn't be specific to webpack. It should be generic so any webpack like tool can generate stats files that confirm to the schema. Django (and other non-JS projects) can then rely on that generic stats file schema to load static assets. This should allow multiple ecosystems to have just a single set of libraries to access static assets generated by any build tool.

What should such a schema for the stats file look like?

owais · 2020-01-08T13:18:48Z

What other features/change would people like to see before we release 1.0?

owais · 2020-01-08T13:34:02Z

I'll create a 1.0 milestone once we have a good idea about what 1.0 should look like.

owais · 2020-01-08T13:59:34Z

One more: #56

melck · 2020-01-09T10:15:07Z

Sounds good but since we decided to only support webpack>4, we have a good opportunity to improve other things and break backward compatibility.

One thing I'd like to do is to default to relative paths and remove the relative path option. We should also remove the path key from stats file and instead add relativePath and absolutePath keys. This will ensure other tools that consume the stats file break in an obvious way.

Do we really need absolute path ? I think path should always be relative from stat file and webpack seems to always avoid absolute path.

Regarding a standard stat file, I think we should capture all the assets and specify for js files if they are entry points or loaded on demand.

Regarding SRI, i have made a working exemple on my branch but i saw after that plugin https://github.com/waysact/webpack-subresource-integrity who do a better job. Maybe we could only use output of this plugin to add integrity to stat file if present.

cscanlin · 2020-02-03T19:57:47Z

+1

#52 (comment)
It would be great to have a standard way to track bundles for all webpack (and other) projects.

If this (or any) format could be made into a standard, the create-react-app team may be inclined to include it as part of that project, which would be a big win for users of django-webpack-loader.
facebook/create-react-app#99 (comment)

jonalxh · 2020-03-13T19:41:35Z

It is very necessary to allow the compressed bundles load, like .gz and .br.

melck · 2020-03-15T15:04:21Z

@jonalxh I will add tests with compression-webpack-plugin to see how it behave and fix it if needed.

@cscanlin The way we track bundles come from webpack api v4 reference. If you think about a case that will not work, tell me and i will add tests to check.

melck · 2020-03-15T18:17:16Z

I have tried compression-webpack-plugin to understand how it works. It seems he only add compressed files to stats.compilation.assets and do not modify chunkGroups. With the plugin option all assets are not compressed because of ratio and treshold. To handle this case, we could modify structure of webpack-stats.json. First add assets entry with all assets generated then simplify chunks entry to refer to a asset name.

Exemple :

I added a test with this webpack-config.json and forced compression on all files:

{
  context: __dirname,
  entry: {
    app1: path.resolve(__dirname, 'fixtures', 'app1.js'),
    appWithAssets: path.resolve(__dirname, 'fixtures', 'appWithAssets.js'),
  },
  output: {
    path: OUTPUT_DIR,
    filename: 'js/[name]-[hash].js',
    publicPath: 'http://localhost:3000/',
  },
  module: {
    rules: [{ test: /\.css$/, use: [MiniCssExtractPlugin.loader, 'css-loader'] }],
  },
  optimization: {
    splitChunks: {
      cacheGroups: {
        vendors: {
          name: 'vendors',
          test: /[\\/]node_modules[\\/]/,
          priority: -10,
          chunks: 'initial',
        },
        commons: {
          name: 'commons',
          test: /[\\/]?commons/,
          enforce: true,
          priority: -20,
          chunks: 'all',
          reuseExistingChunk: true,
        },
        default: {
          name: 'shared',
          reuseExistingChunk: true,
        },
      },
    },
  },
  plugins: [
    new MiniCssExtractPlugin({ filename: 'css/[name]-[hash].css' }),
    new CompressionPlugin({
      filename: '[path].gz[query]',
      test: /\.(js|css)$/,
      threshold: 1,
      minRatio: 1, // Compress all files
      deleteOriginalAssets: false,
    }),
    new CompressionPlugin({
      filename: '[path].br[query]',
      algorithm: 'brotliCompress',
      test: /\.(js|css)$/,
      threshold: 1,
      minRatio: 1, // Compress all files
      deleteOriginalAssets: false,
      compressionOptions: {
        params: {
          [zlib.constants.BROTLI_PARAM_QUALITY]: 11
        }
      }
    }),
    new BundleTrackerPlugin({
      path: OUTPUT_DIR,
      relativePath: true,
      includeParents: true,
    }),
  ],
}

The results of this test give (i forced compression on all files) :

/tmp/wbt-tests-bHBIph/
├── css
│   ├── appWithAssets-7da09931011a19e41c04.css
│   ├── appWithAssets-7da09931011a19e41c04.css.br
│   └── appWithAssets-7da09931011a19e41c04.css.gz
├── js
│   ├── 2-7da09931011a19e41c04.js
│   ├── 2-7da09931011a19e41c04.js.br
│   ├── 2-7da09931011a19e41c04.js.gz
│   ├── app1-7da09931011a19e41c04.js
│   ├── app1-7da09931011a19e41c04.js.br
│   ├── app1-7da09931011a19e41c04.js.gz
│   ├── appWithAssets-7da09931011a19e41c04.js
│   ├── appWithAssets-7da09931011a19e41c04.js.br
│   ├── appWithAssets-7da09931011a19e41c04.js.gz
│   ├── commons-7da09931011a19e41c04.js
│   ├── commons-7da09931011a19e41c04.js.br
│   ├── commons-7da09931011a19e41c04.js.gz
│   ├── vendors-7da09931011a19e41c04.js
│   ├── vendors-7da09931011a19e41c04.js.br
│   └── vendors-7da09931011a19e41c04.js.gz
└── webpack-stats.json

I have change webpack-tracker.json structure who give now :

{
  "status": "done",
  "chunks": {
    "app1": [
      "js/vendors-7da09931011a19e41c04.js",
      "js/commons-7da09931011a19e41c04.js",
      "js/app1-7da09931011a19e41c04.js"
    ],
    "appWithAssets": [
      "js/vendors-7da09931011a19e41c04.js",
      "js/commons-7da09931011a19e41c04.js",
      "css/appWithAssets-7da09931011a19e41c04.css",
      "js/appWithAssets-7da09931011a19e41c04.js"
    ]
  },
  "publicPath": "http://localhost:3000/",
  "assets": [
    {
      "name": "js/commons-7da09931011a19e41c04.js",
      "path": "js/commons-7da09931011a19e41c04.js",
      "publicPath": "http://localhost:3000/js/commons-7da09931011a19e41c04.js"
    },
    {
      "name": "js/vendors-7da09931011a19e41c04.js",
      "path": "js/vendors-7da09931011a19e41c04.js",
      "publicPath": "http://localhost:3000/js/vendors-7da09931011a19e41c04.js"
    },
    {
      "name": "js/2-7da09931011a19e41c04.js",
      "path": "js/2-7da09931011a19e41c04.js",
      "publicPath": "http://localhost:3000/js/2-7da09931011a19e41c04.js"
    },
    {
      "name": "js/app1-7da09931011a19e41c04.js",
      "path": "js/app1-7da09931011a19e41c04.js",
      "publicPath": "http://localhost:3000/js/app1-7da09931011a19e41c04.js"
    },
    {
      "name": "css/appWithAssets-7da09931011a19e41c04.css",
      "path": "css/appWithAssets-7da09931011a19e41c04.css",
      "publicPath": "http://localhost:3000/css/appWithAssets-7da09931011a19e41c04.css"
    },
    {
      "name": "js/appWithAssets-7da09931011a19e41c04.js",
      "path": "js/appWithAssets-7da09931011a19e41c04.js",
      "publicPath": "http://localhost:3000/js/appWithAssets-7da09931011a19e41c04.js"
    },
    {
      "name": "js/commons-7da09931011a19e41c04.js.gz",
      "path": "js/commons-7da09931011a19e41c04.js.gz",
      "publicPath": "http://localhost:3000/js/commons-7da09931011a19e41c04.js.gz"
    },
    {
      "name": "js/2-7da09931011a19e41c04.js.gz",
      "path": "js/2-7da09931011a19e41c04.js.gz",
      "publicPath": "http://localhost:3000/js/2-7da09931011a19e41c04.js.gz"
    },
    {
      "name": "js/vendors-7da09931011a19e41c04.js.gz",
      "path": "js/vendors-7da09931011a19e41c04.js.gz",
      "publicPath": "http://localhost:3000/js/vendors-7da09931011a19e41c04.js.gz"
    },
    {
      "name": "js/app1-7da09931011a19e41c04.js.gz",
      "path": "js/app1-7da09931011a19e41c04.js.gz",
      "publicPath": "http://localhost:3000/js/app1-7da09931011a19e41c04.js.gz"
    },
    {
      "name": "css/appWithAssets-7da09931011a19e41c04.css.gz",
      "path": "css/appWithAssets-7da09931011a19e41c04.css.gz",
      "publicPath": "http://localhost:3000/css/appWithAssets-7da09931011a19e41c04.css.gz"
    },
    {
      "name": "js/appWithAssets-7da09931011a19e41c04.js.gz",
      "path": "js/appWithAssets-7da09931011a19e41c04.js.gz",
      "publicPath": "http://localhost:3000/js/appWithAssets-7da09931011a19e41c04.js.gz"
    },
    {
      "name": "js/2-7da09931011a19e41c04.js.br",
      "path": "js/2-7da09931011a19e41c04.js.br",
      "publicPath": "http://localhost:3000/js/2-7da09931011a19e41c04.js.br"
    },
    {
      "name": "js/commons-7da09931011a19e41c04.js.br",
      "path": "js/commons-7da09931011a19e41c04.js.br",
      "publicPath": "http://localhost:3000/js/commons-7da09931011a19e41c04.js.br"
    },
    {
      "name": "css/appWithAssets-7da09931011a19e41c04.css.br",
      "path": "css/appWithAssets-7da09931011a19e41c04.css.br",
      "publicPath": "http://localhost:3000/css/appWithAssets-7da09931011a19e41c04.css.br"
    },
    {
      "name": "js/app1-7da09931011a19e41c04.js.br",
      "path": "js/app1-7da09931011a19e41c04.js.br",
      "publicPath": "http://localhost:3000/js/app1-7da09931011a19e41c04.js.br"
    },
    {
      "name": "js/vendors-7da09931011a19e41c04.js.br",
      "path": "js/vendors-7da09931011a19e41c04.js.br",
      "publicPath": "http://localhost:3000/js/vendors-7da09931011a19e41c04.js.br"
    },
    {
      "name": "js/appWithAssets-7da09931011a19e41c04.js.br",
      "path": "js/appWithAssets-7da09931011a19e41c04.js.br",
      "publicPath": "http://localhost:3000/js/appWithAssets-7da09931011a19e41c04.js.br"
    }
  ]
}

This structure handle files who are not in chunkGroup, so we have bundled apps and rest of the assets. The choice of taking compressed assets will be made by upstream program (ex: django-webpack-loader).

What do you think about that ?

jonalxh · 2020-03-17T21:48:37Z

You're right, I'm compressing files in the same way, in this case we would talk about the same initial idea so that from the django-webpack-loader we can define what type of file is going to be used, be it .br or .gz

melck · 2020-03-20T14:10:14Z

@owais Any thoughts on this subject ?

owais · 2020-03-20T15:20:16Z

The proposal looks good to me. What would the template and python loader interface be to load different files? Would it change in any way?

melck · 2020-03-21T15:59:34Z

@owais Yes it would change template and python loader. It's clearly a breaking change but it will allow better selection of assets. In my example, webpack compress all files but in real life it will only compress files who have a good ratio and threshold.

Changes for python loader behavior when status done :

Read chunks entry in json file ;
For each asset in each chunk name, check asset name in assets ;
If exist, it load path, publicPath, integrity, etc. for asset.

We can add a option for python loader to priorise selection of compresssed asset. After python loader get list of assets of each chunk, it will look if compressed asset entry exist in asset entry. At first result found, we stop search and use asset details to serve file.

For python loader, we need to add a option COMPRESSED_ASSET_PRIORITY who contains a array of compression by priority (ex: [ 'brotli', 'gzip']). This option will be used to find compressed asset.

owais · 2020-03-21T18:36:10Z

I understand it'll change the implementation. What I meant was if it'll change interface of how the template tags are written or the Python API. If so, I'd like us to discuss and agree on the new interface first. Implementation details can be discussed later as needed.

owais · 2020-03-21T18:51:23Z

For python loader, we need to add a option COMPRESSED_ASSET_PRIORITY

My first reaction is to avoid something like that and let users pick which format they'd prefer in the template.

melck · 2020-03-22T01:23:31Z

For python loader, we need to add a option COMPRESSED_ASSET_PRIORITY

I first reaction is to avoid something like that and let users pick which format they'd prefer in the template.

Good idea 👍

melck mentioned this issue Dec 17, 2019

Support webpack 4 only #53

Merged

owais mentioned this issue Jan 8, 2020

Can plugin's constructor name be changed to 'BundleTrackerPlugin'? #45

Closed

owais mentioned this issue Jan 8, 2020

Gzipped bundles are not being tracked #22

Closed

owais added this to the 1.0.0 milestone Jan 8, 2020

melck mentioned this issue Mar 21, 2020

Features for v1.0.0 #59

Merged

owais closed this as completed in #59 Apr 7, 2020

melck mentioned this issue Apr 11, 2020

1.0.0-alpha.1 output is wrong #61

Closed

pushyamig mentioned this issue Jul 28, 2020

#987 optimizing and compressing webpack assets tl-its-umich-edu/my-learning-analytics#996

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v1.0.0 #52

v1.0.0 #52

melck commented Dec 15, 2019 •

edited

Loading

owais commented Jan 8, 2020

owais commented Jan 8, 2020

owais commented Jan 8, 2020

owais commented Jan 8, 2020

owais commented Jan 8, 2020

melck commented Jan 9, 2020

cscanlin commented Feb 3, 2020

jonalxh commented Mar 13, 2020

melck commented Mar 15, 2020

melck commented Mar 15, 2020 •

edited

Loading

jonalxh commented Mar 17, 2020

melck commented Mar 20, 2020

owais commented Mar 20, 2020

melck commented Mar 21, 2020

owais commented Mar 21, 2020

owais commented Mar 21, 2020 •

edited

Loading

melck commented Mar 22, 2020

v1.0.0 #52

v1.0.0 #52

Comments

melck commented Dec 15, 2019 • edited Loading

owais commented Jan 8, 2020

owais commented Jan 8, 2020

owais commented Jan 8, 2020

owais commented Jan 8, 2020

owais commented Jan 8, 2020

melck commented Jan 9, 2020

cscanlin commented Feb 3, 2020

jonalxh commented Mar 13, 2020

melck commented Mar 15, 2020

melck commented Mar 15, 2020 • edited Loading

jonalxh commented Mar 17, 2020

melck commented Mar 20, 2020

owais commented Mar 20, 2020

melck commented Mar 21, 2020

owais commented Mar 21, 2020

owais commented Mar 21, 2020 • edited Loading

melck commented Mar 22, 2020

melck commented Dec 15, 2019 •

edited

Loading

melck commented Mar 15, 2020 •

edited

Loading

owais commented Mar 21, 2020 •

edited

Loading