wsi-preprocessing-sos-workflow

A simple pipeline built with SoS Workflow that runs wsi-preprocessing on a remote machine.

wsi-preprocessing is a simple library for preprocessing histopathological whole-slide images (WSI) towards deep learning. Check out its repository.

If you do not have a remote machine ready, you can launch a new AWS EC2 instance with ec2-setup-sos-workflow.

Installation

conda

conda create --name YOUR_ENV_NAME --channel conda-forge python=3.6 pyyaml boto3 sos black
# black is optional, it is sort of a development dependency

git clone https://github.com/lucasrla/wsi-preprocessing-sos-workflow

cd wsi-preprocessing-sos-workflow

conda activate YOUR_ENV_NAME

poetry or pip

git clone https://github.com/lucasrla/wsi-preprocessing-sos-workflow

cd wsi-preprocessing-sos-workflow

# create and activate a virtualenv, for example:
pyenv virtualenv YOUR_ENV_NAME && pyenv local YOUR_ENV_NAME

# install the dependencies, either with:
poetry install
# or:
pip install -r requirements.txt

# note, requirements.txt in this repository were generated via:
# poetry export --without-hashes -f requirements.txt -o requirements.txt

Configuration

# edit project.TEMPLATE.yml to match your needs
vim project.TEMPLATE.yml
# and then save it as project.yml

# make sure that ~/.sos/hosts.yml has the path to your pem_file 
# and that the naming of hosts matches project.yml
vim ~/.sos/hosts.yml

Usage

sos run -c project.yml remote.sos -v4

For more tips and tricks on SoS, read the official docs.

You can also have a look at ec2-setup-sos-workflow.

License

This is Free Software distributed under the GNU General Public License v3.0.