Lungs in ct data #24
Closed
Lungs in ct data #24
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…-pipeline-structure 31 improve comments in the new pipeline structure
- Updated pipeline to new format - Added workaround in KITS19 - Added default_factory to DatasetArgs in all pipelines (possibly python version issue in 3.11)
- Split adding labels and ids to separate steps - Moved dataset preparation to abstract function that is called only when source path exists - Enabled not-creating Masks folder if it is set to None (for classification-only datasets)
…ne-for-coronahack [Coronahack] Create pipeline for Coronahack Dataset
andrekomor
requested changes
Dec 28, 2023
config/dataset_masks_config.py
Outdated
| @@ -10,4 +10,7 @@ | |||
| "kidney": 1, | |||
| "kidney_tumor": 2, | |||
| }, | |||
| "Finding_and_Measuring_Lungs_in_CT_Data": { | |||
| "lungs": 255, | |||
There was a problem hiding this comment.
I don't know if this is a problem, but 255 is already used by BrainMET
There was a problem hiding this comment.
I think it's ok because it only refers to source images.
config/dataset_masks_config.yaml
Outdated
There was a problem hiding this comment.
We are not using yamls anymore
config/dataset_uid_config.yaml
Outdated
config/phases_config.yaml
Outdated
config/runner_config.py
Outdated
Comment on lines
19
to
38
| # KITS21Pipeline( | ||
| # path_args={ | ||
| # "source_path": "", | ||
| # "target_path": "./data/", | ||
| # "labels_path": "", | ||
| # }, | ||
| # ), | ||
| # StanfordCOCAPipeline( | ||
| # path_args={ | ||
| # "source_path": "", | ||
| # "target_path": "./data/", | ||
| # "masks_path": "", | ||
| # }, | ||
| # ), | ||
| # StanfordBrainMETPipeline( | ||
| # path_args={ | ||
| # "source_path": "", | ||
| # "target_path": "./data/", | ||
| # }, | ||
| # ), |
There was a problem hiding this comment.
This shouldn't be commented
src/pipelines/base_pipeline.py
Outdated
There was a problem hiding this comment.
Simillar changes already in main. Please merge with main
Comment on lines
45
to
46
| image_folder_name="Images", | ||
| mask_folder_name="Masks", |
There was a problem hiding this comment.
No need to mention it, then defaults will be used
src/steps/convert_tif2png.py
Outdated
Comment on lines
73
to
74
| # root_path = os.path.join(os.path.dirname(os.path.dirname(X[0])), self.mask_folder_name) | ||
| # mask_paths = glob.glob(f"{root_path}/**/*.tif", recursive=True) |
There was a problem hiding this comment.
Remove unused commented code
src/steps/convert_tif2png.py
Outdated
There was a problem hiding this comment.
Please try to refactor step so it is compatible with changes in #34
src/steps/convert_tif2png.py
Outdated
Comment on lines
95
to
138
| if self.mask_folder_name not in img_path: | ||
|
|
||
| phase_id = self.phase_extractor(img_path) | ||
| img_id = self.img_id_extractor(img_path) | ||
| study_id = self.study_id_extractor(img_path) | ||
| if phase_id not in self.phases.keys(): | ||
| return None | ||
| phase_name = self.phases[phase_id] | ||
| new_file_name = f"{self.dataset_uid}_{phase_id}_{study_id}_{img_id}" | ||
| new_file_name = new_file_name.replace(".tif", ".tiff") | ||
| # Changing .tif to .tiff, so images will be readable for PIL | ||
| tiff_path = os.path.join( | ||
| self.target_path, | ||
| f"{self.dataset_uid}_{self.dataset_name}", | ||
| phase_name, | ||
| self.image_folder_name, | ||
| new_file_name, | ||
| ) | ||
| shutil.copy2(img_path, tiff_path) | ||
| new_path = tiff_path.replace(".tiff", ".png") | ||
| try: | ||
| image = PIL.Image.open(img_path) | ||
| invert = True if np.min(np.array(image)) < 0 else False | ||
| image = image.convert("L") | ||
| if invert: | ||
| image = PIL.ImageOps.invert(image) | ||
| image.save(new_path, format="png") | ||
| os.remove(tiff_path) | ||
| except IOError: | ||
| print("Image not found") | ||
| else: | ||
| new_path = img_path.replace(".tif", ".png").replace(".tiff", ".png") | ||
| try: | ||
| # Change .tif to .tiff, so images will be readable for PIL | ||
| tiff_path = img_path.replace(".tif", ".tiff") | ||
| os.rename(img_path, tiff_path) | ||
| img_path = tiff_path | ||
| image = PIL.Image.open(img_path) | ||
| invert = True if np.min(np.array(image)) < 0 else False | ||
| image = image.convert("L") | ||
| if invert: | ||
| image = PIL.ImageOps.invert(image) | ||
| image.save(new_path, format="png") | ||
| os.remove(img_path) |
There was a problem hiding this comment.
Please add some comments. Describe what are the differences between what happens in if statement, and else statement.
The code in lines 116-124, and 132-140 looks identical to me, why do we need to duplicate it?
# Conflicts: # config/dataset_masks_config.py # config/runner_config.py # poetry.lock # src/pipelines/base_pipeline.py
|
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was weird bug with mypy and precommit, so I added one one line to precommit config based on this thread
python/mypy#10632
Please check it.
Pipeline works ok.