Mobile deployment

[aliasboink]

Hi again!

I saw that in the paper on which this is built upon, Supervised Compression for Resource-Constrained Edge Computing Systems from 2021, mentions the deployment of the encoder on mobile devices such at the Raspberry Pi 4 and NVIDIA Jetson TX2.

Is there an example of how someone can deploy the head model on the mobile device and the tail model on the edge server, as to effectively replicate what was done in the study?

[yoshitomo-matsubara]

Hi @banciuadrian ,

Is your question about how we split nn.Module instances of the models in the WACV '22 paper into head and tail models?

For deployment, this repo does not offer such a script at this moment. In the WACV '22 paper, we manually split models into 2 pipelines (i.e., 2 nn.Module instances): 1) model input -> encoder output for mobile device and 2) tail model input (encoder output) -> tail model output for edge server.

Let me know if you need further clarifications.

By the way, I happened to find one of your repos using both sc2bench and torchdistill
https://github.com/banciuadrian/torchdistill_dense_gcn
Like torchdistill, I'd suggest remove sc2bench/ folder in the repo and just do pip3 install sc2bench so that you can make the repo much simpler and easily update sc2bench package by pip when I release next version.

[yoshitomo-matsubara]

Hi @banciuadrian

I'm gathering answers to your questions in one thread.

This just caused some confusion, I could make educated guesses, but nothing I'm certain of. Could you please clarify this aspect? Thank you a lot!

In short, the following part is used at runtime (i.e., after training session).

sc2-benchmark/sc2bench/models/backbone.py

Lines 137 to 141 in a7eaeb8

	if self.bottleneck_updated and not self.training:
	x = self.bottleneck_layer.encode(x)
	if self.analyzes_after_compress:
	self.analyze(x)
	x = self.bottleneck_layer.decode(**x)

and

x = self.bottleneck_layer(x)

is used at training and internally encodes x and decodes the encoded object, which is almost the same as

x = self.bottleneck_layer.encode(x)
x = self.bottleneck_layer.decode(**x)

The main difference between these two processes is whether or not bottleneck_layer does entropy coding, which is used for better compression at runtime.

---

Upon loading this student model, I printed the model and found that the backbone is: (backbone): FeatureExtractionBackbone. Shouldn't it be SplittableResNet with the parameters given in the wrapper splittable_resnet?

For DeepLabv3 with splittable ResNet, splittable_resnet is used as a backbone but decomposed into lower-level modules (and some last layers were removed as DeepLabv3 doesn't use them) to be used by FeatureExtractionBackbone, which is not our backbone meant in the yaml or our paper either.
Note that the DeepLabV3 model (from torchvision) has IntermediateLayerGetter instance as backbone. I found it difficult to introduce encoder-decoder to this module because of its forward implementation.
So I implemented FeatureExtractionBackbone, which is similar to IntermediateLayerGetter but with more flexibility for introducing encoder-decoder

---

This is more of a question to clarify something, there is little doubt here, but I want to make sure. FCNHead the aux_classifier and DeepLabHead the classifier are heads in the "normal sense", i.e., they're heads that come after a backbone, not part of a head-tail model you'd see in split computing. Is my understanding of this correct?

I think that your understanding is correct. And I know that the name of our "head" in our previous papers is getting confusing these days. e.g., Transformer uses heads for attention, vision models like FCN, DeepLab, and more use "head" as last modules in the models. Given that, I'd say these sound more like standard than the concept of "head" used in some of our papers.

Just to clarify, we are terming modules to compress data and deployed on mobile device "encoders" instead of "head" in our recent code and papers.

e.g.,

• SC2: Supervised Compression for Split Computing
• Supervised Compression for Resource-Constrained Edge Computing Systems

[aliasboink]

Hi @yoshitomo-matsubara

I have managed to split the models and send the encoded data using Robotic Operating System (ROS), specifically with rospy.
The answers to the questions were very helpful!
I will mark this as the correct answer and thus close the discussion.

Thank you a lot for the help!

[yoshitomo-matsubara]

Hi @banciuadrian

My pleasure! Feel free to open a new discussion if you find any other questions.

[aliasboink]

Hi @yoshitomo-matsubara

Thank you for the heads up regarding the repository, I've made the suggested changes (to be noted at the moment the repository is really messy and not of any use).

I do have a set of questions I've gathered while trying to split the modules, I hope this isn't too much!

1. In the CompressAI tutorial the forward function follows the pattern encode->bottleneck->decode. Looking at, say, a figure of a model with bottleneck injection this seems to make sense, as you want to send data fit for the bottleneck to reduce the load on the network.
   In sc2-bench I see that the forward function:
   

   sc2-benchmark/sc2bench/models/backbone.py

   Line 133 in a7eaeb8

   def forward(self, x):

   
   follows the pattern of encode->decode after the bottleneck has been updated and the model is not training (is in deployment or testing). This just caused some confusion, I could make educated guesses, but nothing I'm certain of. Could you please clarify this aspect? Thank you a lot!

2. I've loaded the model and ran some scripts. Looking at the model with the intention to modify it (split it) I've stumbled something that caused me confusion:
   

   sc2-benchmark/configs/pascal_voc2012/supervised_compression/entropic_student/deeplabv3_splittable_resnet50-fp-beta1.28_from_deeplabv3_resnet50.yaml

   Line 81 in e326a33

   name: 'splittable_resnet'

   
   Upon loading this student model, I printed the model and found that the backbone is: (backbone): FeatureExtractionBackbone. Shouldn't it be SplittableResNet with the parameters given in the wrapper splittable_resnet?

3. This is more of a question to clarify something, there is little doubt here, but I want to make sure. FCNHead the aux_classifier and DeepLabHead the classifier are heads in the "normal sense", i.e., they're heads that come after a backbone, not part of a head-tail model you'd see in split computing. Is my understanding of this correct?

[yoshitomo-matsubara]

Hi @banciuadrian

I'm answering the questions above.

Like the previous discussion at torchdistill repo, your follow-up comment is posted as an answer.
Next time please "Reply" to the answer if you have follow-up questions or need clarifications for the answer.