AttributeError in Gradio Web Server: 'list' object has no attribute 'text_models'

[tsunghan-wu]

Description:

I followed the instructions to run the WebGUI service in single-model mode by launching the controller, model workers, and Gradio web server. However, when I opened the website, the Gradio web server triggered an error, and the website failed to render correctly.

Error Message and Screenshot:

2024-10-22 21:45:16 | ERROR | stderr | Traceback (most recent call last):
2024-10-22 21:45:16 | ERROR | stderr |   File "/home/patrickwu/miniconda3/envs/fastchat/lib/python3.10/site-packages/gradio/queueing.py", line 624, in process_events
2024-10-22 21:45:16 | ERROR | stderr |     response = await route_utils.call_process_api(
2024-10-22 21:45:16 | ERROR | stderr |   File "/home/patrickwu/miniconda3/envs/fastchat/lib/python3.10/site-packages/gradio/route_utils.py", line 323, in call_process_api
2024-10-22 21:45:16 | ERROR | stderr |     output = await app.get_blocks().process_api(
2024-10-22 21:45:16 | ERROR | stderr |   File "/home/patrickwu/miniconda3/envs/fastchat/lib/python3.10/site-packages/gradio/blocks.py", line 2018, in process_api
2024-10-22 21:45:16 | ERROR | stderr |     result = await self.call_function(
2024-10-22 21:45:16 | ERROR | stderr |   File "/home/patrickwu/miniconda3/envs/fastchat/lib/python3.10/site-packages/gradio/blocks.py", line 1567, in call_function
2024-10-22 21:45:16 | ERROR | stderr |     prediction = await anyio.to_thread.run_sync(  # type: ignore
2024-10-22 21:45:16 | ERROR | stderr |   File "/home/patrickwu/.local/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
2024-10-22 21:45:16 | ERROR | stderr |     return await get_async_backend().run_sync_in_worker_thread(
2024-10-22 21:45:16 | ERROR | stderr |   File "/home/patrickwu/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
2024-10-22 21:45:16 | ERROR | stderr |     return await future
2024-10-22 21:45:16 | ERROR | stderr |   File "/home/patrickwu/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
2024-10-22 21:45:16 | ERROR | stderr |     result = context.run(func, *args)
2024-10-22 21:45:16 | ERROR | stderr |   File "/home/patrickwu/miniconda3/envs/fastchat/lib/python3.10/site-packages/gradio/utils.py", line 846, in wrapper
2024-10-22 21:45:16 | ERROR | stderr |     response = f(*args, **kwargs)
2024-10-22 21:45:16 | ERROR | stderr |   File "/home/patrickwu/FastChat/fastchat/serve/gradio_web_server.py", line 263, in load_demo
2024-10-22 21:45:16 | ERROR | stderr |     return load_demo_single(models, url_params)
2024-10-22 21:45:16 | ERROR | stderr |   File "/home/patrickwu/FastChat/fastchat/serve/gradio_web_server.py", line 235, in load_demo_single
2024-10-22 21:45:16 | ERROR | stderr |     models = context.text_models
2024-10-22 21:45:16 | ERROR | stderr | AttributeError: 'list' object has no attribute 'text_models'

Potential Issue:

In the load_demo function, we're passing a list as the first argument to load_demo_single(models, url_params). However, it appears that this argument should be a Context object, not a list. The error occurs because the list object doesn't have the text_models attribute, which causes the AttributeError.

Steps to Reproduce:

1. Follow the instructions to install the package and run the WebGUI service in single-model mode.
2. Launch the controller, model workers, and Gradio web server.
3. Open the website and observe the error.

[liziniu]

same Issue

[TonyLianLong]

Has this been solved? My temporary solution is to update the load_demo in fastchat/serve/gradio_web_server.py:

def load_demo(url_params, request: gr.Request):
    global models

    ip = get_ip(request)
    logger.info(f"load_demo. ip: {ip}. params: {url_params}")

    if args.model_list_mode == "reload":
        models, all_models = get_model_list(
            controller_url, args.register_api_endpoint_file, vision_arena=False
        )
    
    # Create a Context object with the models
    context = Context()
    context.text_models = models
    context.models = all_models if 'all_models' in locals() else models

    return load_demo_single(context, url_params)