How to use visual automation to send an email - here's an example (for Gmail)

[fazlur10]

I took a ss of outlook icon from my desktop, and saved it in same directory of my tag, and used followig tag
click outlook.png
But my tag do not work it keeps on running, but no outputs are show yet
Should we set up Sikuli before trying visual automation?

[kensoh]

Hi @fazlur10 here's an example sending email through visual automation -
https://github.com/aimakerspace/TagUI-Bricks/tree/master/DBS-Forex-Gmail#tagui-workflow

Oh no need to set up SikuliX, but to install OpenJDK here if there is no Java 64-bit on your system
(can check with java -version on the command prompt) - https://tagui.readthedocs.io/en/latest/setup.html

[paumillet]

Hello @kensoh
I'm facing the same issue: I took screenshots of some web page elements but when I run tagui, it keeps on running but nothing happens.
I'm not able to run the example of sending email through visual automation, it stays on the instruction click chrome_icon.png, even when I try to resize chrome with 125% zoom or change the png file with my own screenshoot.
Do you have any suggestion to make it work?
I'm running tagui v6.14 on ubuntu 20.04
Thanks

[fazlur10]

avoid visual automation and follow generic way https://mail.google.com/mail/u/0/ click Compose wait 5s enter To as ***@***.*** enter Subject as New Products List email_links = arrayUnique.join("\n") if ***@***.***=":sv"]//div') enter ***@***.***=":sv"]//div as `email_links` if ***@***.***=":rc"]') click ***@***.***=":rc"] should write for each different possible cases, bcs in such situations interfaces change when accessing https://mail.google.com/mail/u/0/ 1. not logged in 2. logged in 3. chose google account and enter pw etc. above sample is only for when logged in

…

On Wed, 30 Jun 2021 at 16:21, Pauline ***@***.***> wrote: Hello @kensoh <https://github.com/kensoh> I'm facing the same issue: I took screenshots of some web page elements but when I run tagui, it keeps on running but nothing happens. I'm not able to run the example of sending email through visual automation, it stays on the instruction click chrome_icon.png, even when I try to resize chrome with 125% zoom or change the png file with my own screenshoot. Do you have any suggestion to make it work? I'm running tagui v6.14 on ubuntu 20.04 Thanks — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1052 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOBP5K5PGI64YDSYNJZU443TVLZLHANCNFSM46HOO2CQ> .

[paumillet]

Well, the whole point for me is to actually use visual automation in order to make person not used to code able to write their own process.
Visual automation is a way to make it more accessible.

[kensoh]

Hi @paumillet and @fazlur10,

Oh yes doing using XPath can overcome these challenges, for eg using Chrome extension like SelectorsHub can grab such XPath easily, and using object repositories can hide these complexity away. But the initial work still needs to be done.

For visual automation, it is more visual and easy to understand. But for Linux, it requires additional setup steps here. This is because Linux is actually quite broad in flavours, so each flavour has its own way to install software. Thus the computer vision and OCR components can't be pre-packaged like in Windows and macOS version of TagUI.

I'm guessing maybe there additional steps to installed is not yet done or something broken there. In any case, can you share the logs in tagui/src/tagui.sikuli folder then I can look further what could be causing TagUI to hang there?

[paumillet]

Hi @kensoh,

Thanks for your answer.
I follow the set up steps that where missing but it seems like it's still not working...

Here the logs:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/home/pauline/.Sikulix/Extensions/jython-standalone-2.7.1.jar) to method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of org.python.core.PySystemState
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[tagui] START  - listening for inputs

[tagui] INPUT  - [1] vision setAutoWaitTimeout(300)
[tagui] ACTION - setAutoWaitTimeout(300)
[tagui] OUTPUT - [1] SUCCESS

[tagui] INPUT  - [2] vision wait_timeout = 300
[tagui] ACTION - wait_timeout = 300
[tagui] OUTPUT - [2] SUCCESS

[tagui] INPUT  - [3] click /home/pauline/Documents/RD/RPA/tagui/IRAS-Notice-OCR/#_menu.png
[tagui] ACTION - click /home/pauline/Documents/RD/RPA/tagui/IRAS-Notice-OCR/#_menu.png

I had to stop the process at the end because it was running for a while.

[fazlur10]

but i tried visual automation with windows, still it did not work for me

…

On Thu, 1 Jul 2021 at 08:19, Ken Soh ***@***.***> wrote: Hi @paumillet <https://github.com/paumillet> and @fazlur10 <https://github.com/fazlur10>, Oh yes doing using XPath can overcome these challenges, for eg using Chrome extension like SelectorsHub can grab such XPath easily, and using object repositories can hide these complexity away. But the initial work still needs to be done. For visual automation, it is more visual and easy to understand. But for Linux, it requires additional setup steps here <https://sikulix-2014.readthedocs.io/en/latest/newslinux.html>. This is because Linux is actually quite broad in flavours, so each flavour has its own way to install software. Thus the computer vision and OCR components can't be pre-packaged like in Windows and macOS version of TagUI. I'm guessing maybe there additional steps to installed is not yet done or something broken there. In any case, can you share the logs in tagui/src/tagui.sikuli folder then I can look further what could be causing TagUI to hang there? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1052 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOBP5K3WCPJINOE7GERZ5VLTVPJT7ANCNFSM46HOO2CQ> .

[kensoh]

Hi @fazlur10 can you share your logs here? (tagui\src\tagui.sikuli folder) I assume you already installed OpenJDK. There is also a Windows installer now for TagUI that installs all these - https://tagui.readthedocs.io/en/latest/setup.html

Hi @paumillet I see, from your log it looks like everything is happening ok between TagUI and the SikuliX engine, but somehow when making computer vision action it hangs. This normally is related to OpenCV installation. Is your timezone conducive to join any of our weekly Zoom Q&A calls? It is every Thursday 4-5pm (UTC+8). The issue probably requires some deep troubleshooting. I don't have experience setting up on Linux but I can take a look.