Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fixed backend issues: issue #212 #213

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

Dasux
Copy link

@Dasux Dasux commented Feb 13, 2025

This PR addresses issue #212
Issues in script.sh and server.py have been resolved

Changes made:

  • Replaced all instances of the deprecated jors_winkler module with jaro_winkler_similarity in server.py and main.py to resolve the import error.

  • Implemented a fallback mechanism to force the script to use the CPU instead of the GPU if insufficient VRAM is detected, addressing the CUDA out-of-memory error on devices with limited GPU memory.

  • Set the environment variable os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True" to avoid memory fragmentation and improve GPU memory allocation handling.

NOTE:
Two of the files I've added in this PR.. jaro_wrinkler.py and convert_score_and_distance_functions.py should be replaced from their respective directories... namely:

  1. EduAid/backend/venv/lib/python3.12/site-packages/similarity/jaro_winkler.py
  2. EduAid/backend/venv/lib/python3.12/site-packages/similarity/convert_score_and_distance_functions.py

These changes should fix the reported issues with the server script and improve the overall reliability on systems with limited GPU resources.

@Aditya062003
Copy link
Contributor

Hey, would be great if you could provide an screen capture for the issue resolved!

@Dasux
Copy link
Author

Dasux commented Feb 17, 2025

sure,,,

The script.sh update was already handled in #188 by someone else, so I focused on fixing the jellyfish function issues.

a little more context of the issue

The function jaro_winkler no longer exists in jellyfish—it has been renamed to jaro_winkler_similarity.

image

This caused import issues in multiple files, such as:

  • EduAid/backend/venv/lib/python3.12/site-packages/similarity/jaro_winkler.py

  • EduAid/backend/venv/lib/python3.12/site-packages/similarity/convert_score_and_distance_functions.py

I updated these files to use the correct method name:

image

image

Regarding server.py
I believe I have fixed the issue, but I haven't been able to fully test it due to hardware limitations.

I’m running into a CUDA out-of-memory error—my iGPU has 1.69GB VRAM, with only 41MB free. The script tries to load 96MB, which causes it to fail.

I’ve looked into alternatives, but so far, I haven’t found a workaround.

I think it'd be better if someone else could verify server.py fix since i cant test it properly.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants