pytorch transfer learning and model hosting failed to load model test case

[quantum-fusion]

Your issue may already be reported!
Please search on the issue tracker before creating one.

Pytorch transfer learning and model load failure.

I tried out the transfer learning like shown here (https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html) and this example(https://github.com/pytorch/tutorials/blob/master/beginner_source/transfer_learning_tutorial.py), and then added a save model to the end of the python script. (https://www.dropbox.com/s/jru4p9hbbazm7zn/transfer_learning_tutorial.py?dl=0)

I exported the model and converted to .Mar format (https://www.dropbox.com/s/m29a1h1y0u6haa8/model_save.tar?dl=0)
using this archiver command:
torch-model-archiver --model-name transferlearningmodel --version 1.0 --model-file ~/torchserve/serve/examples/image_classifier/resnet_18/model.py --serialized-file ~/pytorch/tutorials/beginner_source/model_save/transferlearningmodel.pth --export-path model_save --extra-files ~/torchserve/serve/examples/image_classifier/index_to_name.json --handler image_classifier

Once the archiver convert to .MAR was complete I started the torch serve:

(base) MacBook-Pro:~/pytorch/tutorials/beginner_source quantum-fusion$ torchserve --start --ncs --model-store ~/pytorch/tutorials/beginner_source/model_save --models transferlearningmodel.mar

The model did not load successfully, see errors: Load model failed: transferlearningmodel, error: Worker died.

2020-08-16 16:07:01,994 [DEBUG] W-9013-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException
at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1668)
at java.base/java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:435)
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:129)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
2020-08-16 16:07:01,994 [WARN ] W-9013-transferlearningmodel_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: transferlearningmodel, error: Worker died.
2020-08-16 16:07:01,994 [DEBUG] W-9013-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9013-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2020-08-16 16:07:01,994 [INFO ] W-9013-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9013-transferlearningmodel_1.0-stdout
2020-08-16 16:07:01,994 [INFO ] W-9013-transferlearningmodel_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9013-transferlearningmodel_1.0-stderr
2020-08-16 16:07:01,994 [WARN ] W-9013-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9013-transferlearningmodel_1.0-stderr
2020-08-16 16:07:01,994 [WARN ] W-9013-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9013-transferlearningmodel_1.0-stdout
2020-08-16 16:07:01,994 [INFO ] W-9013-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9013 in 55 seconds.
2020-08-16 16:07:02,003 [INFO ] KQueueEventLoopGroup-4-29 org.pytorch.serve.wlm.WorkerThread - 9004 Worker disconnected. WORKER_STARTED
2020-08-16 16:07:02,003 [DEBUG] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2020-08-16 16:07:02,003 [DEBUG] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException
at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1668)
at java.base/java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:435)
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:129)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
2020-08-16 16:07:02,004 [WARN ] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: transferlearningmodel, error: Worker died.
2020-08-16 16:07:02,004 [DEBUG] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9004-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2020-08-16 16:07:02,004 [INFO ] W-9004-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9004-transferlearningmodel_1.0-stdout
2020-08-16 16:07:02,004 [INFO ] W-9004-transferlearningmodel_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9004-transferlearningmodel_1.0-stderr
2020-08-16 16:07:02,004 [WARN ] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9004-transferlearningmodel_1.0-stderr
2020-08-16 16:07:02,004 [WARN ] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9004-transferlearningmodel_1.0-stdout
2020-08-16 16:07:02,004 [INFO ] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9004 in 55 seconds.

(base) MacBook-Pro:/pytorch/tutorials/beginner_source quantum-fusion$ torchserve --stop
TorchServe has stopped.
2020-08-16 16:07:22,985 [INFO ] KQueueEventLoopGroup-2-2 org.pytorch.serve.ModelServer - Management model server stopped.
2020-08-16 16:07:22,985 [INFO ] KQueueEventLoopGroup-2-1 org.pytorch.serve.ModelServer - Inference model server stopped.
(base) MacBook-Pro:/pytorch/tutorials/beginner_source quantum-fusion$ 2020-08-16 16:07:25,194 [INFO ] main org.pytorch.serve.ModelServer - Torchserve stopped.

[dhaniram-kshirsagar]

@quantum-fusion There aren't enough details in given log hence will try executing this and let you know.
In the meantime, please provide detail logs/info if any

[quantum-fusion]

@dhaniram-kshirsagar I tested the generated .MAR file using the Pytorch Archiver, and then ran another test this time using the multi-model-server ( https://github.com/awslabs/multi-model-server ) . The transcript file is here (https://www.dropbox.com/s/zhh87q0mtmys2zh/modelserverLog.txt?dl=0). The transcript file provides a lot more data, and also leads me to believe that there may have been an issue with how the .MAR file was generated using the Pytorch archiver.

@dhaniram-kshirsagar I exported the model and converted to .Mar format (https://www.dropbox.com/s/m29a1h1y0u6haa8/model_save.tar?dl=0)
using this archiver command:
torch-model-archiver --model-name transferlearningmodel --version 1.0 --model-file ~/torchserve/serve/examples/image_classifier/resnet_18/model.py --serialized-file ~/pytorch/tutorials/beginner_source/model_save/transferlearningmodel.pth --export-path model_save --extra-files ~/torchserve/serve/examples/image_classifier/index_to_name.json --handler image_classifier

@dhaniram-kshirsagar Do you see anything wrong with how the .PTH file was saved?

save checkpoint

path = "/Users/hottelet/pytorch/tutorials/beginner_source/model_save/transferlearningmodel.pth"
torch.save(model_conv.state_dict(),path)

[prashantsail]

Hi @quantum-fusion

Analysis

1. I have used the transferlearningmodel.mar file that you have shared in dropbox.
2. I see following error in the logs -

2020-08-17 10:19:22,136 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "/home/ubuntu/.local/lib/python3.6/site-packages/ts/model_service_worker.py", line 176, in <module>
2020-08-17 10:19:22,136 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     worker.run_server()
2020-08-17 10:19:22,136 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "/home/ubuntu/.local/lib/python3.6/site-packages/ts/model_service_worker.py", line 148, in run_server
2020-08-17 10:19:22,136 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.handle_connection(cl_socket)
2020-08-17 10:19:22,137 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "/home/ubuntu/.local/lib/python3.6/site-packages/ts/model_service_worker.py", line 112, in handle_connection
2020-08-17 10:19:22,137 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     service, result, code = self.load_model(msg)
2020-08-17 10:19:22,137 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "/home/ubuntu/.local/lib/python3.6/site-packages/ts/model_service_worker.py", line 85, in load_model
2020-08-17 10:19:22,137 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     service = model_loader.load(model_name, model_dir, handler, gpu, batch_size)
2020-08-17 10:19:22,137 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "/home/ubuntu/.local/lib/python3.6/site-packages/ts/model_loader.py", line 117, in load
2020-08-17 10:19:22,137 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     model_service.initialize(service.context)
2020-08-17 10:19:22,137 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "/home/ubuntu/.local/lib/python3.6/site-packages/ts/torch_handler/base_handler.py", line 50, in initialize
2020-08-17 10:19:22,137 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.model = self._load_pickled_model(model_dir, model_file, model_pt_path)
2020-08-17 10:19:22,138 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "/home/ubuntu/.local/lib/python3.6/site-packages/ts/torch_handler/base_handler.py", line 83, in _load_pickled_model
2020-08-17 10:19:22,138 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     model.load_state_dict(state_dict)
2020-08-17 10:19:22,138 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -   File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1045, in load_state_dict
2020-08-17 10:19:22,138 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.__class__.__name__, "\n\t".join(error_msgs)))
2020-08-17 10:19:22,138 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - RuntimeError: Error(s) in loading state_dict for ImageClassifier:
2020-08-17 10:19:22,138 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -         size mismatch for fc.weight: copying a param with shape torch.Size([2, 512]) from checkpoint, the shape in current model is torch.Size([1000, 512]).
2020-08-17 10:19:22,138 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -         size mismatch for fc.bias: copying a param with shape torch.Size([2]) from checkpoint, the shape in current model is torch.Size([1000]).

3. Looking closely at your example transfer_learning_tutorial.py and the hymenoptera_data (which is used for training and prediction) : There are ONLY "2" classes - 'ants' and 'bees'
4. Where as the ResNet class from torchvision package is by default initialized with 'num_classes=1000'
5. This causes the fc.weight & fc.bias shape mismatch error as seen in the logs above.

Solution

• Initialize the model with correct number of classes (num_classes) and also use a custom handler to tweak the max result classes(topk) for inference (The default 'image_classifier' handler cannot be used as it assumes at least 5 result classes)
• Please follow the steps mentioned below to create a new transferlearningmodel.mar, it will help you successfully serve the model and predict with images.

1. Create a directory and add following files to it -
   1. transferlearningmodel.pth
   2. model.py

      from torchvision.models.resnet import ResNet, BasicBlock
      
      class ImageClassifier(ResNet):
          def __init__(self):
              super(ImageClassifier, self).__init__(BasicBlock, [2,2,2,2], num_classes = 2)

   3. handler.py

      from ts.torch_handler.image_classifier import ImageClassifier as ImgClsfr
      
      class ImageClassifier(ImgClsfr):
          topk = 2

   4. index_to_name.json

      {"0": ["id01", "ant"], "1": ["id02", "bee"]}

2. Create a new *.mar file

   torch-model-archiver --model-name transferlearningmodel --version 1.0 \
   --serialized-file ./transferlearningmodel.pth \
   --model-file ./model.py \
   --handler ./handler.py \
   --extra-files ./index_to_name.json
   

3. Copy the newly created transferlearningmodel.mar to your model store directory and start torchserve

   torchserve --start --ncs --model-store ./modelstore --models transferlearningmodel.mar
   

4. Predict with images

   curl http://127.0.0.1:8080/predictions/transferlearningmodel -T test_image1.jpg
   curl http://127.0.0.1:8080/predictions/transferlearningmodel -T test_image2.jpg
   

Notes:

1. This discussion in the pytorch forum helped me.
2. Read more about custom handlers here.

[dhaniram-kshirsagar]

This should be added to ts guide #215

[quantum-fusion]

@dhaniram-kshirsagar I have tried the following steps.

1. Executed ./beginner_source/transfer_learning_tutorial.py , and it produced ./model_save/transferlearning.pth

2. I copied the ./model_save/transferlearning.pth to ./beginner_source/testTransferLearning

3. created the model.py , handler.py , and index_to_name.json files as you had described, and place them in ./beginner_source/testTransferLearning

4. Created the new mar file in testTransferLearning directory
   torch-model-archiver --model-name transferlearningmodel --version 1.0
   --serialized-file ./transferlearningmodel.pth
   --model-file ./model.py
   --handler ./handler.py
   --extra-files ./index_to_name.json

5. Executed torch serve from within testTransferLearning directory
   torchserve --start --ncs --model-store ./ --models transferlearningmodel.mar

6. The W-9000 error messages disappeared, but
   there are new error messages that prevent the curl command to execute successfully
   see ./beginner_source/testTransferLearning/logs/ts_log.log

7. The complete tar file of the project is here (https://www.dropbox.com/s/hn4hp0oxrb6lf9h/beginner_source.tar?dl=0)

8. I would like to know how you had arrived to index_to_name.json, because I do not understand the naming convention. How do you know that id01, and id02 are the keys, and the values are ant, and bee? Is there a published standard online? The directory example actually had many ants and bees. Please help clarify how you come to this conclusion answer?
   {"0": ["id01", "ant"], "1": ["id02", "bee"]}

[dhaniram-kshirsagar]

Looks like you are using torchserve 0.1.1 release [per logs on dropbox], can you please upgrade your torchserve to the latest and try again.

[quantum-fusion]

@dhaniram-kshirsagar torch-model-archiver and torch serve was upgraded to 0.2.0 . I am trying again. The MAR format has to be 0.2.0

[quantum-fusion]

@dhaniram-kshirsagar Are we not expecting the result to be a tiger cat, like the torchserve example? This transfer learning experiment was supposed to add Ants and Bees as a tensor pattern, but not replace the Tiger cat with an Ant or Bee. Please let me know why the index_to_name.json class names were only Ant for 0, and Bee for 1. Also, please tell me why those index class names replace the more than 1000 class names already in the Resnet18 class.

The result is as follows:
curl http://127.0.0.1:8080/predictions/transferlearningmodel -T kitten.jpg
{
"ant": 0.6434735655784607,
"bee": 0.3565264046192169
}

torchserve --start --ncs --model-store ./ --models transferlearningmodel.mar
(venv) (base) MacBook-Pro:~/pytorch/tutorials/beginner_source/testTransferLearning quantum-fusion$ 2020-08-17 15:28:14,035 [INFO ] main org.pytorch.serve.ModelServer -
Torchserve version: 0.2.0
TS Home: /Users/hottelet/venv/lib/python3.8/site-packages
Current directory: /Users/hottelet/pytorch/tutorials/beginner_source/testTransferLearning
Temp directory: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T/
Number of GPUs: 0
Number of CPUs: 16
Max heap size: 8192 M
Python executable: /Users/hottelet/venv/bin/python3
Config file: N/A
Inference address: http://127.0.0.1:8080
Management address: http://127.0.0.1:8081
Metrics address: http://127.0.0.1:8082
Model Store: /Users/hottelet/pytorch/tutorials/beginner_source/testTransferLearning
Initial Models: transferlearningmodel.mar
Log dir: /Users/hottelet/pytorch/tutorials/beginner_source/testTransferLearning/logs
Metrics dir: /Users/hottelet/pytorch/tutorials/beginner_source/testTransferLearning/logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 16
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Prefer direct buffer: false
Custom python dependency for model allowed: false
Metrics report format: prometheus
Enable metrics API: true
2020-08-17 15:28:14,066 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: transferlearningmodel.mar
2020-08-17 15:28:15,029 [INFO ] main org.pytorch.serve.archive.ModelArchive - eTag 3d0f2c6fc0fb415e9c6983bb760d6b55
2020-08-17 15:28:15,037 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model transferlearningmodel
2020-08-17 15:28:15,037 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model transferlearningmodel
2020-08-17 15:28:15,037 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model transferlearningmodel loaded.
2020-08-17 15:28:15,038 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: transferlearningmodel, count: 16
2020-08-17 15:28:15,052 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: KQueueServerSocketChannel.
2020-08-17 15:28:15,172 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://127.0.0.1:8080
2020-08-17 15:28:15,173 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: KQueueServerSocketChannel.
2020-08-17 15:28:15,174 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://127.0.0.1:8081
2020-08-17 15:28:15,174 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: KQueueServerSocketChannel.
2020-08-17 15:28:15,176 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082
Model server started.
2020-08-17 15:28:15,193 [WARN ] pool-2-thread-1 org.pytorch.serve.metrics.MetricCollector - worker pid is not available yet.
2020-08-17 15:28:15,357 [INFO ] W-9013-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9013
2020-08-17 15:28:15,357 [INFO ] W-9006-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9006
2020-08-17 15:28:15,358 [INFO ] W-9012-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9012
2020-08-17 15:28:15,358 [INFO ] W-9013-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46009
2020-08-17 15:28:15,358 [INFO ] W-9012-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]45999
2020-08-17 15:28:15,358 [INFO ] W-9006-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46004
2020-08-17 15:28:15,358 [INFO ] W-9013-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,358 [INFO ] W-9010-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9010
2020-08-17 15:28:15,359 [INFO ] W-9006-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,359 [INFO ] W-9004-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9004
2020-08-17 15:28:15,358 [INFO ] W-9012-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,359 [INFO ] W-9010-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46013
2020-08-17 15:28:15,359 [DEBUG] W-9013-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9013-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,359 [INFO ] W-9012-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,359 [DEBUG] W-9010-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9010-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,359 [INFO ] W-9006-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,359 [INFO ] W-9013-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,359 [INFO ] W-9010-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,359 [DEBUG] W-9012-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9012-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,359 [INFO ] W-9004-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46005
2020-08-17 15:28:15,359 [DEBUG] W-9006-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9006-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,360 [INFO ] W-9010-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,360 [INFO ] W-9004-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,360 [DEBUG] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9004-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,360 [INFO ] W-9004-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,361 [INFO ] W-9009-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9009
2020-08-17 15:28:15,361 [INFO ] W-9009-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46001
2020-08-17 15:28:15,361 [INFO ] W-9009-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,361 [DEBUG] W-9009-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9009-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,361 [INFO ] W-9009-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,362 [INFO ] W-9003-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9003
2020-08-17 15:28:15,363 [INFO ] W-9003-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46008
2020-08-17 15:28:15,363 [INFO ] W-9003-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,363 [DEBUG] W-9003-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9003-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,363 [INFO ] W-9003-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,364 [INFO ] pool-2-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692495
2020-08-17 15:28:15,364 [INFO ] W-9014-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9014
2020-08-17 15:28:15,364 [INFO ] pool-2-thread-1 TS_METRICS - DiskAvailable.Gigabytes:3222.298984527588|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692495
2020-08-17 15:28:15,365 [INFO ] W-9014-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46010
2020-08-17 15:28:15,365 [INFO ] W-9003-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9003
2020-08-17 15:28:15,365 [INFO ] pool-2-thread-1 TS_METRICS - DiskUsage.Gigabytes:10.480133056640625|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692495
2020-08-17 15:28:15,365 [INFO ] W-9006-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9006
2020-08-17 15:28:15,365 [INFO ] W-9014-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,365 [INFO ] W-9013-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9013
2020-08-17 15:28:15,365 [INFO ] W-9009-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9009
2020-08-17 15:28:15,365 [INFO ] W-9014-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,365 [INFO ] pool-2-thread-1 TS_METRICS - DiskUtilization.Percent:0.3|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692495
2020-08-17 15:28:15,365 [INFO ] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9004
2020-08-17 15:28:15,365 [INFO ] W-9012-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9012
2020-08-17 15:28:15,365 [INFO ] W-9010-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9010
2020-08-17 15:28:15,365 [INFO ] pool-2-thread-1 TS_METRICS - MemoryAvailable.Megabytes:11430.35546875|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692495
2020-08-17 15:28:15,365 [DEBUG] W-9014-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9014-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,366 [INFO ] W-9014-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9014
2020-08-17 15:28:15,366 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUsed.Megabytes:17296.34765625|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692495
2020-08-17 15:28:15,366 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUtilization.Percent:65.1|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692495
2020-08-17 15:28:15,367 [INFO ] W-9011-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9011
2020-08-17 15:28:15,367 [INFO ] W-9011-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46000
2020-08-17 15:28:15,368 [INFO ] W-9011-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,368 [DEBUG] W-9011-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9011-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,368 [INFO ] W-9011-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9011
2020-08-17 15:28:15,368 [INFO ] W-9011-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,373 [INFO ] W-9002-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9002
2020-08-17 15:28:15,373 [INFO ] W-9002-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46012
2020-08-17 15:28:15,373 [INFO ] W-9002-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,373 [DEBUG] W-9002-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,373 [INFO ] W-9002-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,373 [INFO ] W-9002-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9002
2020-08-17 15:28:15,374 [INFO ] W-9005-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9005
2020-08-17 15:28:15,374 [INFO ] W-9005-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46002
2020-08-17 15:28:15,374 [INFO ] W-9005-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,374 [DEBUG] W-9005-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9005-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,374 [INFO ] W-9005-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,374 [INFO ] W-9005-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9005
2020-08-17 15:28:15,376 [INFO ] W-9001-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9001
2020-08-17 15:28:15,377 [INFO ] W-9007-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9007
2020-08-17 15:28:15,377 [INFO ] W-9001-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46014
2020-08-17 15:28:15,377 [INFO ] W-9001-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,377 [DEBUG] W-9001-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9001-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,377 [INFO ] W-9008-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9008
2020-08-17 15:28:15,377 [INFO ] W-9001-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9001
2020-08-17 15:28:15,377 [INFO ] W-9001-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,378 [INFO ] W-9007-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46011
2020-08-17 15:28:15,378 [INFO ] W-9008-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46007
2020-08-17 15:28:15,378 [INFO ] W-9007-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,378 [DEBUG] W-9007-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9007-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,378 [INFO ] W-9008-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,378 [DEBUG] W-9008-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9008-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,378 [INFO ] W-9008-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,378 [INFO ] W-9007-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9007
2020-08-17 15:28:15,378 [INFO ] W-9007-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,378 [INFO ] W-9008-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9008
2020-08-17 15:28:15,379 [INFO ] W-9015-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9015
2020-08-17 15:28:15,379 [INFO ] W-9015-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46006
2020-08-17 15:28:15,379 [INFO ] W-9015-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,379 [DEBUG] W-9015-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9015-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,379 [INFO ] W-9015-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9015
2020-08-17 15:28:15,379 [INFO ] W-9015-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,382 [INFO ] W-9010-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9010.
2020-08-17 15:28:15,382 [INFO ] W-9001-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9001.
2020-08-17 15:28:15,382 [INFO ] W-9007-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9007.
2020-08-17 15:28:15,382 [INFO ] W-9012-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9012.
2020-08-17 15:28:15,382 [INFO ] W-9004-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9004.
2020-08-17 15:28:15,382 [INFO ] W-9008-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9008.
2020-08-17 15:28:15,382 [INFO ] W-9003-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9003.
2020-08-17 15:28:15,383 [INFO ] W-9006-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9006.
2020-08-17 15:28:15,382 [INFO ] W-9009-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9009.
2020-08-17 15:28:15,383 [INFO ] W-9011-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9011.
2020-08-17 15:28:15,383 [INFO ] W-9013-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9013.
2020-08-17 15:28:15,383 [INFO ] W-9002-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9002.
2020-08-17 15:28:15,383 [INFO ] W-9014-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9014.
2020-08-17 15:28:15,383 [INFO ] W-9005-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9005.
2020-08-17 15:28:15,383 [INFO ] W-9015-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9015.
2020-08-17 15:28:15,393 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9000
2020-08-17 15:28:15,393 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46003
2020-08-17 15:28:15,394 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2020-08-17 15:28:15,394 [DEBUG] W-9000-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-transferlearningmodel_1.0 State change null -> WORKER_STARTED
2020-08-17 15:28:15,394 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.8.3
2020-08-17 15:28:15,394 [INFO ] W-9000-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9000
2020-08-17 15:28:15,396 [INFO ] W-9000-transferlearningmodel_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /var/folders/60/l9x5s_456sx9nspbd1l90_fm0000gn/T//.ts.sock.9000.
2020-08-17 15:28:17,481 [INFO ] W-9007-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2015
2020-08-17 15:28:17,482 [DEBUG] W-9007-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9007-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,483 [INFO ] W-9007-transferlearningmodel_1.0 TS_METRICS - W-9007-transferlearningmodel_1.0.ms:2437|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,498 [INFO ] W-9008-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2064
2020-08-17 15:28:17,498 [DEBUG] W-9008-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9008-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,498 [INFO ] W-9008-transferlearningmodel_1.0 TS_METRICS - W-9008-transferlearningmodel_1.0.ms:2452|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,501 [INFO ] W-9014-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2034
2020-08-17 15:28:17,501 [DEBUG] W-9014-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9014-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,501 [INFO ] W-9014-transferlearningmodel_1.0 TS_METRICS - W-9014-transferlearningmodel_1.0.ms:2454|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,529 [INFO ] W-9010-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2060
2020-08-17 15:28:17,530 [DEBUG] W-9010-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9010-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,530 [INFO ] W-9010-transferlearningmodel_1.0 TS_METRICS - W-9010-transferlearningmodel_1.0.ms:2483|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,550 [INFO ] W-9000-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2080
2020-08-17 15:28:17,551 [DEBUG] W-9000-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,551 [INFO ] W-9000-transferlearningmodel_1.0 TS_METRICS - W-9000-transferlearningmodel_1.0.ms:2508|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,554 [INFO ] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2085
2020-08-17 15:28:17,554 [DEBUG] W-9004-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9004-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,554 [INFO ] W-9004-transferlearningmodel_1.0 TS_METRICS - W-9004-transferlearningmodel_1.0.ms:2508|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,579 [INFO ] W-9005-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2110
2020-08-17 15:28:17,579 [DEBUG] W-9005-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9005-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,579 [INFO ] W-9005-transferlearningmodel_1.0 TS_METRICS - W-9005-transferlearningmodel_1.0.ms:2533|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,580 [INFO ] W-9013-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2111
2020-08-17 15:28:17,580 [DEBUG] W-9013-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9013-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,580 [INFO ] W-9013-transferlearningmodel_1.0 TS_METRICS - W-9013-transferlearningmodel_1.0.ms:2533|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,588 [INFO ] W-9002-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2123
2020-08-17 15:28:17,588 [DEBUG] W-9002-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,588 [INFO ] W-9002-transferlearningmodel_1.0 TS_METRICS - W-9002-transferlearningmodel_1.0.ms:2542|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,589 [INFO ] W-9015-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2123
2020-08-17 15:28:17,589 [DEBUG] W-9015-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9015-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,589 [INFO ] W-9015-transferlearningmodel_1.0 TS_METRICS - W-9015-transferlearningmodel_1.0.ms:2542|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,590 [INFO ] W-9006-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2121
2020-08-17 15:28:17,590 [DEBUG] W-9006-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9006-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,590 [INFO ] W-9006-transferlearningmodel_1.0 TS_METRICS - W-9006-transferlearningmodel_1.0.ms:2544|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,593 [INFO ] W-9011-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2127
2020-08-17 15:28:17,594 [DEBUG] W-9011-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9011-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,594 [INFO ] W-9011-transferlearningmodel_1.0 TS_METRICS - W-9011-transferlearningmodel_1.0.ms:2547|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,597 [INFO ] W-9001-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2129
2020-08-17 15:28:17,597 [DEBUG] W-9001-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9001-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,597 [INFO ] W-9001-transferlearningmodel_1.0 TS_METRICS - W-9001-transferlearningmodel_1.0.ms:2551|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,599 [INFO ] W-9009-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2133
2020-08-17 15:28:17,599 [DEBUG] W-9009-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9009-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,599 [INFO ] W-9009-transferlearningmodel_1.0 TS_METRICS - W-9009-transferlearningmodel_1.0.ms:2552|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,600 [INFO ] W-9012-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2133
2020-08-17 15:28:17,600 [DEBUG] W-9012-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9012-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,600 [INFO ] W-9012-transferlearningmodel_1.0 TS_METRICS - W-9012-transferlearningmodel_1.0.ms:2553|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:28:17,602 [INFO ] W-9003-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2168
2020-08-17 15:28:17,602 [DEBUG] W-9003-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - W-9003-transferlearningmodel_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED
2020-08-17 15:28:17,602 [INFO ] W-9003-transferlearningmodel_1.0 TS_METRICS - W-9003-transferlearningmodel_1.0.ms:2556|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692497
2020-08-17 15:29:15,275 [INFO ] pool-2-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692555
2020-08-17 15:29:15,276 [INFO ] pool-2-thread-1 TS_METRICS - DiskAvailable.Gigabytes:3222.301498413086|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692555
2020-08-17 15:29:15,276 [INFO ] pool-2-thread-1 TS_METRICS - DiskUsage.Gigabytes:10.480133056640625|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692555
2020-08-17 15:29:15,276 [INFO ] pool-2-thread-1 TS_METRICS - DiskUtilization.Percent:0.3|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692555
2020-08-17 15:29:15,276 [INFO ] pool-2-thread-1 TS_METRICS - MemoryAvailable.Megabytes:10217.51953125|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692555
2020-08-17 15:29:15,276 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUsed.Megabytes:18487.84765625|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692555
2020-08-17 15:29:15,276 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUtilization.Percent:68.8|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692555
2020-08-17 15:29:52,949 [INFO ] W-9007-transferlearningmodel_1.0-stdout MODEL_METRICS - PredictionTime.Milliseconds:128.38|#ModelName:transferlearningmodel,Level:Model|#hostname:Henrys-MacBook-Pro-2.local,requestID:2dae8120-3e67-4bd8-8e5e-883bf0dccfd6,timestamp:1597692592
2020-08-17 15:29:52,949 [INFO ] W-9007-transferlearningmodel_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 129
2020-08-17 15:29:52,950 [INFO ] W-9007-transferlearningmodel_1.0 ACCESS_LOG - /127.0.0.1:61986 "PUT /predictions/transferlearningmodel HTTP/1.1" 200 137
2020-08-17 15:29:52,950 [INFO ] W-9007-transferlearningmodel_1.0 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:null
2020-08-17 15:29:52,951 [DEBUG] W-9007-transferlearningmodel_1.0 org.pytorch.serve.wlm.Job - Waiting time ns: 124418, Backend time ns: 132329691
2020-08-17 15:30:15,277 [INFO ] pool-2-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692615
2020-08-17 15:30:15,278 [INFO ] pool-2-thread-1 TS_METRICS - DiskAvailable.Gigabytes:3222.3090019226074|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692615
2020-08-17 15:30:15,278 [INFO ] pool-2-thread-1 TS_METRICS - DiskUsage.Gigabytes:10.480133056640625|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692615
2020-08-17 15:30:15,278 [INFO ] pool-2-thread-1 TS_METRICS - DiskUtilization.Percent:0.3|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692615
2020-08-17 15:30:15,278 [INFO ] pool-2-thread-1 TS_METRICS - MemoryAvailable.Megabytes:10185.83203125|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692615
2020-08-17 15:30:15,278 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUsed.Megabytes:18536.828125|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692615
2020-08-17 15:30:15,278 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUtilization.Percent:68.9|#Level:Host|#hostname:Henrys-MacBook-Pro-2.local,timestamp:1597692615

[quantum-fusion]

@dhaniram-kshirsagar I had a similar set of results using the BentoML transfer learning as well, and while the REST api had a similar format, the transfer learning also produced the wrong sets of results, which invalidated the performance of the model. (jjmachan/resnet-bentoml#1).

@dhaniram-kshirsagar Can you please help with explaining the transfer learning results, and should we not expect to see a Tiger Cat as the result? Transfer learning should add patterns, not delete them.

[quantum-fusion]

@prashantsail Do you see the results not showing a Tiger Cat? I don't think this is what we were expecting.

[quantum-fusion]

@prashantsail look at results below, I used the index_to_name.json from the torch serve repo, and now we see wrong results
@dhaniram-kshirsagar look at results below, I used the index_to_name.json from the torch serve repo, and now see wrong results.

@prashantsail It may be that we have the wrong format for the index_to_name.json that does not include the ants and bees
@dhaniram-kshirsagar It may be that we have the wrong format for the index_to_name.json that does not include the ants and bees

---

Step1:
torch-model-archiver --model-name transferlearningmodel --version 1.0 --model-file ./model.py --serialized-file ./transferlearningmodel.pth --extra-files /Users/hottelet/torchserve/serve/examples/image_classifier/index_to_name.json --handler ./handler.py

Step2:
torchserve --start --ncs --model-store ./ --models transferlearningmodel.mar

Results:
curl http://127.0.0.1:8080/predictions/transferlearningmodel -T kitten.jpg
{
"tench": 0.6434735059738159,
"goldfish": 0.3565264046192169
}(base) MacBook-Pro:/Keras quantum-fusion$ curl http://127.0.0.1:8080/predictions/transferlearningmodel -T seal1.jpg
{
"tench": 0.9586480855941772,
"goldfish": 0.04135190695524216
}(base) MacBook-Pro:/Keras quantum-fusion$ curl http://127.0.0.1:8080/predictions/transferlearningmodel -T seal2.jpg
{
"tench": 0.8636642098426819,
"goldfish": 0.1363358348608017
}(base) MacBook-Pro:~/Keras quantum-fusion$ curl http://127.0.0.1:8080/predictions/transferlearningmodel -T seal3.jpg
{
"tench": 0.8587694764137268,
"goldfish": 0.1412305384874344

[prashantsail]

@quantum-fusion

Referring to transfer_learning_tutorial.py
For creating the model, you have started with a Resnet18 model and used feature extraction for transfer learning scenario.

1. Resnet18's "Fully Connected" layer has "1000" out_features (classes)
2. In the line 312 - model_conv.fc = nn.Linear(num_ftrs, 2)
   You have reinitialized the model's "Fully Connected" layer to "2" out_features.
3. Further, you have optimized this layer and trained the model again with 'hymenoptera_data' (dataset with only 2 classes - ants & bees).

index_to_name.json

1. This file is merely used for formatting the prediction API response (user convenience)
2. Labels in the file are configured as per the data your model is trained with.
   1. index_to_name.json (and other examples) from the torch serve repo, is as per the ImageNet 2012 Classification Dataset.
   2. index_to_name.json in your case, has to be as per the hymenoptera dataset
3. Providing --extra-files ./index_to_name.json file is OPTIONAL.
4. If not specified, your prediction API response would look something like this -

   {
       "0": 0.6434735059738159,
       "1": 0.3565264046192169
   }

To clarify further -
✓ The model that you have created is intended for classification of images into 2 classes only - 'ants' or 'bees'
✕ It is NOT going to classify over "1002" classes (1000 classes from the original ResNet and the 2 that you have newly trained)