Streamlit serving

[mike0sv]

TODOs:

• multiple methods
• list model element distillation
• binary payloads
• better interface?
• tests

[mike0sv]

To test you can run:

from mlem.api import serve
from mlem.contrib.streamlit.server import StreamlitServer
from mlem.core.metadata import save


def main():
    save(lambda x: x + 1, "mdl2", sample_data=0)
    serve("mdl2", StreamlitServer())


if __name__ == '__main__':
    main()

[mike0sv]

It was quite a journey, but I finally ran streamlit+pytorch in docker with images

Some stuff I found out: mac m1 is aarch64, and tochvision binaries on pypi do not have image.so shared library for some reason (pytorch/vision#5919) since 0.10, which is too old for my example

I tried building amd64 image, but watchdog lib does not work with this setup (it's optional though, so I fixed it with pip uninstall watchdog && touch watchdog.py && PYTHONPATH=. sh run.sh inside the container).

[codecov]

Codecov Report

Base: 87.24% // Head: 86.70% // Decreases project coverage by -0.54% ⚠️

Coverage data is based on head (58e290b) compared to base (ff340ad).
Patch coverage: 52.59% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #512      +/-   ##
==========================================
- Coverage   87.24%   86.70%   -0.55%     
==========================================
  Files          97       99       +2     
  Lines        8821     8948     +127     
==========================================
+ Hits         7696     7758      +62     
- Misses       1125     1190      +65     

Impacted Files	Coverage Δ
mlem/contrib/streamlit/_template.py	0.00% <0.00%> (ø)
mlem/ext.py	88.28% <ø> (ø)
mlem/contrib/streamlit/server.py	70.83% <70.83%> (ø)
mlem/runtime/client.py	84.68% <77.77%> (-0.76%)	⬇️
mlem/core/data_type.py	92.72% <100.00%> (+0.01%)	⬆️
mlem/runtime/server.py	88.53% <100.00%> (+0.22%)	⬆️
mlem/utils/entrypoints.py	86.36% <100.00%> (-0.85%)	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[aguschin]

Trying your example:

In [1]: from mlem.api import serve
   ...: from mlem.contrib.streamlit.server import StreamlitServer
   ...: from mlem.core.metadata import save
   ...:
   ...:
   ...: def main():
   ...:     save(lambda x: x + 1, "mdl2", sample_data=0)
   ...:     serve("mdl2", StreamlitServer())
   ...:
   ...:
   ...: if __name__ == '__main__':
   ...:     main()
   ...:
INFO:     Started server process [42653]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
INFO:     127.0.0.1:58074 - "GET / HTTP/1.1" 307 Temporary Redirect
INFO:     127.0.0.1:58074 - "GET /docs HTTP/1.1" 200 OK
INFO:     127.0.0.1:58074 - "GET /openapi.json HTTP/1.1" 200 OK

  You can now view your Streamlit app in your browser.

  URL: http://0.0.0.0:80

  For better performance, install the Watchdog module:

  $ xcode-select --install
  $ pip install watchdog

INFO:     127.0.0.1:58089 - "GET /interface.json HTTP/1.1" 200 OK
INFO:     127.0.0.1:58090 - "GET /interface.json HTTP/1.1" 200 OK
2022-12-09 20:48:06.914 Uncaught app exception
Traceback (most recent call last):
  File "/Users/aguschin/.local/share/virtualenvs/mlem-Utz6DvOn/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 564, in _run_script
    exec(code, module.__dict__)
  File "/private/var/folders/tv/l60j0x050p536g3bh8g2w1n80000gn/T/mlem_streamlit_script_psg1c3au/script.py", line 40, in <module>
    augment, arg_model_aug = augment_model(arg_model)
  File "/Users/aguschin/Git/iterative/mlem/mlem/contrib/streamlit/server.py", line 35, in augment_model
    for name, f in model.__fields__.items()
AttributeError: type object 'int' has no attribute '__fields__'

[aguschin]

Got few issues looking the same with different models, will re-check them once you fix this :)

[aguschin]

When request via streamlit fails, it prints something like

HTTPError: 500 Server Error: Internal Server Error for url: http://0.0.0.0:8080/predict
Traceback:
File "/Users/aguschin/.local/share/virtualenvs/mlem-Utz6DvOn/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 564, in _run_script
    exec(code, module.__dict__)
File "/private/var/folders/tv/l60j0x050p536g3bh8g2w1n80000gn/T/mlem_streamlit_script_v6wsv2qt/script.py", line 64, in <module>
    response = getattr(client, method_name)(
File "/Users/aguschin/Git/iterative/mlem/mlem/runtime/client.py", line 153, in __call__
    out = self.call_method(self.name, data, return_raw)
File "/Users/aguschin/Git/iterative/mlem/mlem/runtime/client.py", line 201, in _call_method
    ret.raise_for_status()
File "/Users/aguschin/.local/share/virtualenvs/mlem-Utz6DvOn/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

but underneath, FastAPI fails with a different error. This is strange, maybe we eventually will need to either redirect failures, or use MLEM model directly instead of forwarding this to FastAPI, which seems more reasonable to me. For now should work though :)