Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Details loaded even for images already in the MD5 list #3140

Closed
ProtagNeptune opened this issue Apr 27, 2024 · 5 comments
Closed

Details loaded even for images already in the MD5 list #3140

ProtagNeptune opened this issue Apr 27, 2024 · 5 comments
Assignees
Labels
Milestone

Comments

@ProtagNeptune
Copy link
Sponsor

When you batch download and it gets a URL with the md5 already in it why should it need to load the details if the file already exists on the md5 database and in the hard drive with the Path/Folder/Filename?
It keeps throwing [Info] Details limit reached (HTTP 503). New try. and slow the batch download by pointlessly loading details when it's just not needed, the file already exists so it should just start the next download on the list.

@Bionus Bionus changed the title Details limit reached (HTTP 503). New try. Details loaded even for images already in the MD5 list Apr 28, 2024
@Bionus Bionus self-assigned this Apr 28, 2024
@Bionus Bionus added this to the 7.12.0 milestone Apr 28, 2024
@solipsis-project
Copy link

There are situations where a user may want to load the details, even if the md5 is already in the database and the file is already on the hard drive at that path.

For instance, I have database hooks that run for each image that insert that image's metadata into a custom DB. If I download the same file from different sources, I don't want that process to be skipped: I want my custom DB to have a separate entry for each source that contains the file.

I'm still learning how to use Grabber, but I'm worried that skipping loading details when an image is already in the md5 list might skip these DB commands, and I want to make sure that won't happen.

@Bionus
Copy link
Owner

Bionus commented Apr 29, 2024

@solipsis-project in those cases, you can simply set your download settings always download the file. But today, the file will be skipped anyway after loading the details, and the SQL commands won't run. This is just an optimization.

@ProtagNeptune
Copy link
Sponsor Author

ProtagNeptune commented May 3, 2024

@Bionus Nope, c6b22e2 didn't fix it so I'm guessing that my metadata setting is still somehow forcing it to load the details even though it's already in the MD5 list and on the disk.
image
image
image
XMP:TagsList

javascript:
'artist/'+artist.replace(/([,;])+/g,'_').replace(/([_]{2,})+/g,'_').replace(/(@#&)+/g,';artist/').toLowerCase()+';'+
'copyright/'+copyright.replace(/([,;])+/g,'_').replace(/([_]{2,})+/g,'_').replace(/(@#&)+/g,';copyright/').toLowerCase()+';'+
'character/'+character.replace(/([,;])+/g,'_').replace(/([_]{2,})+/g,'_').replace(/(@#&)+/g,';character/').toLowerCase()+';'+
'meta/'+meta.replace(/([,;])+/g,'_').replace(/([_]{2,})+/g,'_').replace(/(@#&)+/g,';meta/').toLowerCase()+';'+
'model/'+model.replace(/([,;])+/g,'_').replace(/([_]{2,})+/g,'_').replace(/(@#&)+/g,';model/').toLowerCase()+';'+
'general/'+general.replace(/([,;])+/g,'_').replace(/([_]{2,})+/g,'_').replace(/(@#&)+/g,';general/').toLowerCase()+';'+
'rating/'+rating+';'+
'website/'+website

@Bionus
Copy link
Owner

Bionus commented May 5, 2024

Seems to be because of the "same dir" MD5 setting. Pushed a second fix, sorry about that 👍

@ProtagNeptune
Copy link
Sponsor Author

Yep, 70073d6 fixed this bug for me. 👌

@Bionus Bionus modified the milestones: 7.13.0, 7.12.0 May 12, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants