Running RPA-Python in Docker, tagui rpa_python chrome: not found - pending some checks

[richylyq]

using the #140 (comment) as my reference point for hosting my project with RPA-Python, i am encountering these issues right after

r.init()

[RPA][ERROR] - following happens when starting TagUI...
sh: 1: /home/user/.tagui/src/tagui rpa_python chrome: not found   

r.init(headless_mode = True)

[RPA][ERROR] - following happens when starting TagUI...
sh: 1: /home/user/.tagui/src/tagui rpa_python headless: not found   

What could have gone wrong here? My chrome has been installed in /usr/bin

[fi-do]

Hey @richylyq,

I tried to replicate you problem to get a better view of it. But I'am not able to build an image out of the docker file. Can you give me instructions to build it?

Greetings,
fi-do

[richylyq]

Hey @richylyq,

I tried to replicate you problem to get a better view of it. But I'am not able to build an image out of the docker file. Can you give me instructions to build it?

Greetings,
fi-do

hey @fi-do, here's my dockerfile for your reference

# pull official base image
FROM python:3.6.10-slim

# set environment variables
ENV TZ Asia/Singapore
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Install libraries and Google Chrome.
RUN apt-get update \
    && apt-get install -y unzip wget gnupg python3-pip python3-venv php \
    && wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
    && apt-get update \
    && apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 \
    --no-install-recommends \
    && rm -rf /var/lib/apt/lists/*

# set work directory
WORKDIR /usr/src

# copy scripts
COPY ./src/requirements.txt /usr/src/requirements.txt

# install dependencies
RUN apt-get update && apt-get install -y \
    && rm -rf /var/lib/apt/lists/*

RUN set -eux \
    && pip3 install --upgrade pip setuptools wheel \
    && pip3 install -r requirements.txt \
    && rm -rf /root/.cache/pip

# install php
RUN apt-get update \
    && apt-get install php -y

# Add a user for running applications.
RUN groupadd -r user && useradd -r -g user -G audio,video user \
    && mkdir -p /home/user \
    && chown -R user:user /home/user

# copy source
COPY ./src /usr/src

RUN chown -R user:user /usr/src

USER user

ENTRYPOINT ["bash", "entrypoint.sh"]

With Regards,
richylyq

[kensoh]

Adding on to @fi-do's reply, @richylyq it looks like your Docker file does not install the RPA for Python package. But assuming it is installed, the error message seems to suggest that the file /home/user/.tagui/src/tagui is not found. This is strange because during r.init(), if this file is not found, it will auto call r.setup() to download and install TagUI to that folder.

Some ideas to try will be deleting this folder /home/user/.tagui and try running r.init() again.

You can also try running r.setup() yourself to force an installation to see if any error messages appear.

[kensoh]

You can also try running below command manually (create an empty rpa_python file), to see if any error happen.

/home/user/.tagui/src/tagui rpa_python chrome

[richylyq]

Hey @kensoh, I've tried out the above debug methods but to no avail. I'll be putting Docker on hold for the moment to setup rpa on my local machine but I am facing another issue
I tried to run the codes from the RPA for Python x Colab.ipynb but it always hangs at this portion with rpa_python and tagui_local.js being generated r.debug(True) isn't showing any output as well

[RPA][INFO] - setting up TagUI for use in your Python environment
[RPA][INFO] - downloading TagUI (~200MB) and unzipping to below folder...
[RPA][INFO] - /home/user
[RPA][INFO] - done. syncing TagUI with stable cutting edge version
[RPA][INFO] - TagUI now ready for use in your Python environment
[RPA][INFO] - visual automation (optional) requires special setup on Linux,
[RPA][INFO] - see the link below to install OpenCV and Tesseract libraries
[RPA][INFO] - https://sikulix-2014.readthedocs.io/en/latest/newslinux.html

when i tried ending the process, the same error occurred was shown

r.init()

[RPA][ERROR] - following happens when starting TagUI...
sh: 1: /home/user/.tagui/src/tagui rpa_python chrome: not found   

r.init(headless_mode = True)

[RPA][ERROR] - following happens when starting TagUI...
sh: 1: /home/user/.tagui/src/tagui rpa_python headless: not found   

[kensoh]

Hi @richylyq from your logs above it looks like the /home/user/.tagui/src/tagui command is not found.

Can you verify if there is a folder .tagui under /home/user? From terminal you can run below to see if the file exist?

ls -l /home/user/.tagui/src/tagui

If the folder and file is not there, there might be some permissions issue to write to your machine. But one thing puzzling I don't understand is from your logs when this line shows [RPA][INFO] - done. syncing TagUI with stable cutting edge version it means it detected that the folder and file has been downloaded and unzipped successfully.

[richylyq]

hi @kensoh, the file exists -rwxr-xr-x 1 user user 22708 Jul 30 11:03 /home/user/.tagui/src/tagui
🤯

[kensoh]

Maybe some issue with the user you are using to run Python and the owner user of that folder.

You can check more what is the issue with the Python process using following -

import os
print(os.path.isfile('/home/user/.tagui/src/tagui'))

And from Python itself try to run directly and see what happens -

import os
os.system('/home/user/.tagui/src/tagui rpa_python')

[richylyq]

import os
print(os.path.isfile('/home/user/.tagui/src/tagui'))

this returned True

import os
os.system('/home/user/.tagui/src/tagui rpa_python')

and for this, i've gotten

Terminated
36608

[kensoh]

What happens when you type this from terminal?

/home/user/.tagui/src/tagui rpa_python

[kensoh]

After you try above, try below and see what happens -

/home/user/.tagui/src/tagui /home/user/.tagui/src/samples/1_yahoo

I assume you have followed the steps in https://colab.research.google.com/drive/13bQO6G_hzE1teX35a3NZ4T5K-ICFFdB5?usp=sharing#scrollTo=tQ4BGdSNACgP which includes installing PHP if your Linux doesn't have PHP.

[richylyq]

both of them returned Terminated and yeap, i followed the colab notebook for and made sure to install PHP already

[kensoh]

Oh if both returned terminated when run from your terminal, it looks like some strange issue with upstream TagUI in your env.

See if you can join the weekly Zoom Q&A on Thursday to troubleshoot with screen sharing and remote control -

aisingapore/TagUI#914

[richylyq]

noted on that. Could it be an Ubuntu version mismatch (just guessing tho it should not affect running TagUI)

[richylyq]

hey @kensoh i tried installing phantomjs as per #190 (comment)
and i tried running /home/user/.tagui/src/tagui rpa_python and got this in return

START - automation started - Wed Aug 04 2021 10:45:23 GMT+0800 (+08)

LIVE MODE - type done to quit

then for /home/user/.tagui/src/tagui /home/user/.tagui/src/samples/1_yahoo the result is

PHP Fatal error:  Uncaught TypeError: count(): Argument #1 ($var) must be of type Countable|array, bool given in /home/user/.tagui/src/tagui_parse.php:37
Stack trace:
#0 {main}
  thrown in /home/user/.tagui/src/tagui_parse.php on line 37
Unable to load script /home/user/.tagui/src/samples/1_yahoo.js; check file syntax

[kensoh]

Hi @richylyq can you check what is your php version with php -v? I think on v5 and v7 it works.

[richylyq]

it's v8 currently, i will try to rollback to v7 instead first

[kensoh]

I see.. Before you roll back to v7, can you try deleting the file below -

/home/user/.tagui/src/samples/tagui_local.csv and after that

run /home/user/.tagui/src/tagui /home/user/.tagui/src/samples/1_yahoo ?

I want to see other than this error message, is there any other blockers to troubleshoot.

[richylyq]

after removing the tagui_local.csv and running the 1_yahoo it managed to run the automation following the error

START - automation started - Wed Aug 04 2021 13:03:37 GMT+0800 (+08)
https://ca.yahoo.com - Yahoo

NOTE - this sample may not work for EU users due to website changes for GDPR
EU users may have to modify script to first click on announcement OK button

type search-box as github
show search-box
Search query
click search-button
ERROR - cannot find search-button

[kensoh]

Thanks for your details, looks like the TagUI engine is running ok, can you try run below to see if any problem with Chrome?

/home/user/.tagui/src/tagui /home/user/.tagui/src/samples/1_yahoo chrome

Also, try running below in the folder which the rpa_python file is generated.

/home/user/.tagui/src/tagui rpa_python chrome

Above would be exactly what happens when you do r.init(), hopefully that gives the clue what is happening.

[richylyq]

i tried both of the commands above but they just got stuck for quite a bit without any value return or response.

anw, based on the video, am i correct to say that whenever r.init() is called, it kills all existing rpa processes but for my case since the program is stuck at r.init(), therefore i won't be able to run even the /home/user/.tagui/src/tagui /home/user/.tagui/src/samples/1_yahoo

2021-08-04.16-32-13.mp4

[kensoh]

From your observations, it sounds like there is some blocker with integration to Chrome. An issue I encountered before a due to company network policy, it does not allow TagUI to connect to the websocket connection and thus end up with an infinite loop.

This is the code block in tagui/src/tagui file with that loop.

# wait until chrome is ready with websocket url for php thread
while true; do
ws_url="$(curl -s localhost:9222/json | grep -A 1 '"url": "about:blank"' | grep webSocketDebuggerUrl | cut -d'"' -f 4)"
if [ -n "$ws_url" ]; then break; fi; done

When you run the command /home/user/.tagui/src/tagui /home/user/.tagui/src/samples/1_yahoo chrome from the terminal and it hangs. You can try checking above curl command progressively to see what's wrong.

Alternatively, if you could join the Weekly Zoom Q&A we can look into it together. This is issue with upstream TagUI in your environment and requires me to troubleshoot directly on your laptop -
aisingapore/TagUI#914