Model gets uploaded to S3 when deploying fully local inference model

[johann-petrak]

I tried to test code for deploying a PyTorchModel on SageMaker fully locally doing something like:

I have also created the file ~/.sagemaker/config.yaml as described in https://sagemaker.readthedocs.io/en/stable/overview.html#local-mode

sess = LocalSession()
sess.config = {'local': {'local_code': True}}

model = PyTorchModel(
    model_data="model1.tar.gz",
    role=SOMEROLE, 
    framework_version="1.8.1",  
    py_version="py3",
    entry_point="inference.py",
    )
model.sagemaker_session = sess

predictor = model.deploy(
    instance_type="local", 
    initial_instance_count=1,
    deserializer=JSONDeserializer(),
    serializer=JSONSerializer(),
)

It turns out that this will upload my model to an S3 bucket every time the deploy method is run even though everything is running completely locally.

If testing this completely locally, and if the correct Docker images have been pulled beforehand, then this should not even need an internet connection, but definitely not waste time uploading the model and filling my S3 space with all those model files.

See also #2451

[johann-petrak]

Any idea how to prevent this?

[johann-petrak]

See

sagemaker-python-sdk/src/sagemaker/model.py

Line 1114 in 9e7b4b5

if self.sagemaker_session.local_mode and local_code:

(invoked by model.prepare_container_def) where the upload is skipped if fully local,

BUT for PyTorchModel, prepare_container_def is overriden and always uploads the model:
see

sagemaker-python-sdk/src/sagemaker/pytorch/model.py

Line 239 in 9e7b4b5

self._upload_code(deploy_key_prefix, repack=self._is_mms_version())

[johann-petrak]

Another reason why this bug is really annoying is that there seems to be a request timeout which kicks in when the model.tar.gz file is large and the upload internet speed is not fast enough to upload the model within that request timeout, making it impossible to use the SDK for testing on such a machine.

[raghu-ramesha]

Issue resolved, please use the latest local mode - https://sagemaker.readthedocs.io/en/stable/overview.html?highlight=local%20mode#local-mode