Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Slow file uploading after meteor upgrade #852

Open
valerymagadeeva opened this issue Sep 25, 2022 · 18 comments
Open

Slow file uploading after meteor upgrade #852

valerymagadeeva opened this issue Sep 25, 2022 · 18 comments
Labels
help wanted Need More Info question

Comments

@valerymagadeeva
Copy link

valerymagadeeva commented Sep 25, 2022 •
edited
Loading

Hello! I updated my meteor application to meteor 2.7 (and meteor-files to 2.3.0) and as a result I have a strange issue - the file uploading takes too long time, it uploads extremely slow. (the difference between the old version and the new one is about 30 seconds for 40mb file).
I tried to use both of DDP and HTTP transports and nothing helps. I noticed that when I used HTTP to upload files the meteor server response took at least a second for every chunk. So I guess the problem is somewhere between the client side and the server side.
To upload file from client i use insert() method with chunkSize equals to "dynamic" value. The server side code is the FilesCollection constructor:

const Uploads = new FilesCollection({
    collectionName: 'preupload',
    allowClientCode: false,
    debug: false,
    protected: true,
    onBeforeUpload(file) {},
    onAfterUpload(uploadedFile) {}
})
  • Meteor-Files version - 2.3.0;
  • Meteor version - 2.7;
  • Browser/OS - Google Chrome/MacOS;
  • Seems like that might be server side issue;

Client logs:
[FilesCollection] [insert()]

vue-composition-api.mjs:2246 [FilesCollection] [FileUpload] [constructor]
vue-composition-api.mjs:2246 onBeforeUpload => Mon Sep 26 2022 13:38:11 GMT+0400 (Samara Standard Time)
vue-composition-api.mjs:2246 [FilesCollection] [insert] using WebWorkers
upload.js:368 loadFile ccsetup592.bin (1).zip: 95912.81884765625 ms
vue-composition-api.mjs:2246 [FilesCollection] [UploadInstance] [sendEOF] false
vue-composition-api.mjs:2246 [FilesCollection] [UploadInstance] [end] ccsetup592.bin (1).zip
upload.js:319 insert ccsetup592.bin (1).zip: 96609.42993164062 ms

Server logs(last part) (do not pay attention to warning, i just need console.time for time measurements):

I20220926-09:39:42.742(0)? [FilesCollection] [Upload] [DDP] Got #141/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:42.742(0)? onBeforeUpload => 2022-09-26T09:39:42.742Z
W20220926-09:39:42.743(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:43.262(0)? [FilesCollection] [Upload] [DDP] Got #142/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:43.263(0)? onBeforeUpload => 2022-09-26T09:39:43.262Z
W20220926-09:39:43.263(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:43.888(0)? [FilesCollection] [Upload] [DDP] Got #143/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:43.888(0)? onBeforeUpload => 2022-09-26T09:39:43.888Z
W20220926-09:39:43.888(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:44.825(0)? [FilesCollection] [Upload] [DDP] Got #144/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:44.826(0)? onBeforeUpload => 2022-09-26T09:39:44.825Z
W20220926-09:39:44.826(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:45.649(0)? [FilesCollection] [Upload] [DDP] Got #145/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:45.649(0)? onBeforeUpload => 2022-09-26T09:39:45.649Z
W20220926-09:39:45.650(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:46.173(0)? [FilesCollection] [Upload] [DDP] Got #146/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:46.174(0)? onBeforeUpload => 2022-09-26T09:39:46.173Z
W20220926-09:39:46.174(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:46.954(0)? [FilesCollection] [Upload] [DDP] Got #147/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:46.955(0)? onBeforeUpload => 2022-09-26T09:39:46.954Z
W20220926-09:39:46.955(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:47.503(0)? [FilesCollection] [Upload] [DDP] Got #148/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:47.503(0)? onBeforeUpload => 2022-09-26T09:39:47.503Z
W20220926-09:39:47.504(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:48.121(0)? [FilesCollection] [Upload] [DDP] Got #149/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:48.121(0)? onBeforeUpload => 2022-09-26T09:39:48.121Z
W20220926-09:39:48.121(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:48.192(0)? [FilesCollection] [Upload] [DDP] Got #-1/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:48.193(0)? onBeforeUpload => 2022-09-26T09:39:48.192Z
W20220926-09:39:48.193(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:48.195(0)? [FilesCollection] [Upload] [finish(ing)Upload] -> assets/app/uploads/preupload/sQW9M6qt4WaLbzefJ.zip
I20220926-09:39:48.211(0)? [FilesCollection] [Upload] [finish(ed)Upload] -> assets/app/uploads/preupload/sQW9M6qt4WaLbzefJ.zip
I20220926-09:39:48.212(0)? file uploading1: 1:36.508 (m:ss.mmm)

Thank you in advance!
@dr-dimitru
Copy link
Member

Hello @valerymagadeeva ,

Please update your OP following our ISSUE_TEMPLATE. And post logs with debug enabled

@valerymagadeeva
Copy link
Author

@dr-dimitru updated!

@dr-dimitru
Copy link
Member

@valerymagadeeva thanks, let's start with question:

  1. Why you upload ccsetup592.bin but file saved as ZIP (sQW9M6qt4WaLbzefJ.zip)?

@valerymagadeeva
Copy link
Author

The file's full name is ccsetup592.bin (1).zip - it's zip archive and it should have such extension. Anyway I tried to upload a binary file - the result is the same

@dr-dimitru
Copy link
Member

@valerymagadeeva Do you zip the file during or after upload?

@dr-dimitru
Copy link
Member

@valerymagadeeva to make sure this isn't caused your development environment please download, launch locally, and test uploads via this app (runs on the latest meteor and meteor-files)

@valerymagadeeva
Copy link
Author

@dr-dimitru I unzip the file after uploading in the other part of our system. The problem is not connected to zip file. I tried your demo app and got the same result

@dr-dimitru
Copy link
Member

@valerymagadeeva do you get a good speed when uploading at files.veliov.com ?

@valerymagadeeva
Copy link
Author

@dr-dimitru nope, almost the same result (maybe 10 seconds faster). While in the old version of meteor I have uploaded the file for 45-50 seconds (vs 1m 20 - 1m30 seconds in your demo and my app with the new version of meteor)

@dr-dimitru
Copy link
Member

@valerymagadeeva loads very local, any other device to test on?

I'm getting 10MB/s, which is above my upload speed test
Screen Shot 2022-09-26 at 2 51 30 PM
Screen Shot 2022-09-26 at 2 53 33 PM

@valerymagadeeva
Copy link
Author

Screenshot 2022-09-26 at 16 03 03
I have lower speed when trying to upload via your demo app. But my internet speed is higher
Screenshot 2022-09-26 at 16 04 02

@dr-dimitru
Copy link
Member

dr-dimitru commented Sep 26, 2022 •
edited
Loading

  1. But you're saying speed was higher before last update? Including on demo website?
  2. What speed do you have locally?

@valerymagadeeva
Copy link
Author

  1. idk if the speed was higher on your demo website. Now i have two git branches with file uploading - the first one with old Meteor version (1.11 unfortunately and the file is uploading for 40-50 seconds) and the second one with the new version.
  2. What do you mean under local speed? My hard drive write speed?

@dr-dimitru
Copy link
Member

TLTD; we need clear reproduction steps in order to identify an issue

@valerymagadeeva
Copy link
Author

What info can i provide you?

@dr-dimitru
Copy link
Member

@valerymagadeeva can not come up with something specific, as I'm not sure how to localize your issue

@valerymagadeeva
Copy link
Author

@dr-dimitru ok, I'll try to provide something specific to reproduce the issue. But it seems I have a poor upload speed (according to the information from the demo app). Idk why and im sure it's not connected to my internet or something

@dr-dimitru
Copy link
Member

Does anyone else experiencing this? Or have ever experienced?

# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
help wanted Need More Info question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dr-dimitru @valerymagadeeva