bug: loosen dependency versions

[jaidisido]

Environment Details

Please indicate the following details about the environment in which you found the bug:

• SDV version: 1.0.0
• Python version: 3.8
• Operating System: Mac

Error Description

Is there a particular reason why many of SDV dependency ranges are so tight and restrictive?
Python convention is to pin dependencies at major but to allow minor version upgrades.

In my case I am unable to install sdv in my environment because ctgan is pinned to a restrictive minor version 'ctgan>=0.7.1,<0.8'

This causes a dependency resolution hell with other packages. For instance, installing sdv with black>=23.0 is causing the below:

Steps to reproduce

We use poetry instead of pip for dependency management. Installing the latest version of sdv and black raises the above.

[npatki]

Hi @jaidisido, someone just filed an issue yesterday about compatibility issues with black. I'm curious what project you're working on and what you're using this library for?

To answer your Qs:

Is there a particular reason why many of SDV dependency ranges are so tight and restrictive?

The SDV's dependencies include
(a) internal libraries that are within the overall SDV ecosystem, such as CTGAN, RDT, etc.
(b) external, popular libraries that are needed for data science, such as numpy or pandas

When building the SDV 1.0, we needed to use specific functionality from the internal libraries in (a), hence the lower-bound on CTGAN. We're looking forward to updating our dependency strategy going forward, so this feedback is useful!

For instance, installing sdv with black>=23.0 is causing the below:

This particular issue is caused by CTGAN setting a restrictive dependency on the external packaging library. We have filed CTGAN Issue 276 to look into this.

In the meantime, I wonder if you'd be able to lower the version of black to 22.0 for your project?

[jaidisido]

Thanks for the reply @npatki, I am glad to hear it's being worked on.

My team looks after the AWS SDK for pandas library. I was simply playing with SDV for now and when attempting to add it to our pyproject.toml as an optional dependency, the error was raised by poetry.

Downgrading black might solve the issue. But even then the poetry resolver is taking too long to resolve the dependencies. This comment seem to mention the same issue but with pip. That being said, the long resolution time might be due to a different issue (resolution tree being too large perhaps).

I imagine these issues might be alleviated once you update your dependency strategy as you mentioned.

[npatki]

I'm closing this issue in favor of the CTGAN issue. This will resolve the immediate conflict.

For the longer term strategy, I have filed #1368 for tracking purposes.