Adjust image hit percentage - I believe it's not possible, some ideas to check

[tiagovasconcelosp]

Hello, is it possible to adjust the percentage of correctness of the images? I have a problem, where rpa does not find the image. Even though this image is 100% correct with what is displayed on the screen. Like width x height and pixel.

I believe that there must be a calculation to define a hit pattern and so I would like to regulate it.

[fi-do]

Hey @tiagovasconcelosp ,

I was trying to reconstruct your issue. Can you provide a code snippet to experience that issue?
To answer your question directly, maybe you can work with the function r.vision,

Greets,
fi-do

[tiagovasconcelosp]

Hi @fi-do

Here's an example of what I'm doing. Notice that I open a program and then I make a loop to wait for this time for the program to open, checking for existence with an image x button.

Ignore the comments, I was trying to do the same thing differently to try to get around the problem.

In order to proceed with the opening of the system.

In this time of waiting until he appears, he doesn't think with rpa. Sometimes it works and sometimes it doesn't. So I described the need above.

I'll try to exemplify with the image and the system.

class Auth:

def __init__(self, r, caminhoExe, logging, caminhoPrints, caminhoImages, dataForm, atividade):
    self.Login(r, caminhoExe, logging, caminhoPrints, caminhoImages, dataForm, atividade)

def espera(self, r, imagem, logging, extra = 0, invertido = False):

    espera = 0 
    while not r.present(imagem):
        r.wait(1)
        espera += 1
        # print(str(espera) + 's')
        logging.info(str(espera) + 's')
    # r.wait(extra)

def Login(self, r, caminhoExe, logging, caminhoPrints, caminhoImages, dataForm, atividade):

    try:

        try:
            Sys("start " + caminhoExe)
        except:
            Sys("start C:\Saj\sajsg5\sajsg5.exe")

        # self.espera(r, caminhoImages + 'entrar.png', logging)
        logging.info('SAJ iniciado.')

        # imagem = caminhoImages + 'entrar.png';
        # espera = 10
        # while True:
        #     r.wait(espera)
        #     espera += 5
        #     print(str(espera) + 's')
        #     logging.info(str(espera) + 's')
        #
        #     if r.present(imagem):
        #         break

        r.wait(15)
        logging.info('Inciando autenticacao.')

        r.type(caminhoImages + 'usuario.png', dataForm['login'])

        if atividade == 'Decurso de prazo SAJPG':
            r.click(caminhoImages + 'senha.png')
            r.click(caminhoImages + 'logo-softplan.png')

            r.wait(1)

            esp=0
            while r.present(caminhoImages + 'lotacao_2.png'):
                r.wait(1)
                esp += 1
                print(esp)
                logging.info(str(esp) + 's')

            r.type(caminhoImages + 'senha-2.png', dataForm['pass'])
            r.wait(1)

        else:
            r.wait(1)
            r.type(caminhoImages + 'senha.png', dataForm['pass'])
            r.wait(1)

        # Usado quando havia autenticação manual
        # Remover do código
        # r.wait(tempoAguardar)

        # Verifica se foi clicado em entrar
        if r.present(caminhoImages + 'entrar.png'):
            r.click(caminhoImages + 'entrar.png')
            r.wait(10)

        # Fecha mesagem de fonte
        if r.present(caminhoImages + 'ok.png'):
            r.click(caminhoImages + 'ok.png')
            r.wait(5)

        # Fecha pesquisa
        if r.present(caminhoImages + 'responder-depois.png'):
            r.click(caminhoImages + 'responder-depois.png')
            r.wait(1)

        logging.info("Realizando autenticacao.")

    except Exception as e:

        # image = Print(firefox, caminhoPrints)

        logging.exception('Falha realizar autenticacao.')
        logging.info('Finalizando o robo.')
        logging.info(repr(e))
        logging.shutdown()
        os._exit(0)
        sys.exit(0)

[tiagovasconcelosp]

@fi-do Cropped images used for rpa

[tiagovasconcelosp]

Image of the system that rpa works on. Sorry for the image quality, I took a screenshot of a video

[tiagovasconcelosp]

@fi-do Notice the cropping of the image, you should find it perfectly.

[kensoh]

Adding on, some possible causes could be -

1. if it is Windows, try changing display zoom % to 100% or the recommended % to see if it helps
2. if is on Mac, try checking if it is due to Retina display issue and the solution here
3. if it is due to the app taking too long to load and appear, you can use r.wait(30), for example, to have a longer timeout

I don't think the match % can be set as a parameter, for the underlying computer vision engine used (SikuliX).

[tiagovasconcelosp]

@kensoh Thanks for the help, I understand these mentioned cases. But it's none of those.

It would be interesting to know about the possibility of these parameters but I already imagined that they didn't have it.

My specific problem was due to the list of libraries related to pyinstaller, luckily I was able to identify the problem and solve it.

When I did the conversion, it somehow affected the functioning of rpa/ocr

[kensoh]

I see.. Yes I confirmed that the underlying SikuliX engine does not support adjusting the match % - https://sikulix-2014.readthedocs.io/en/latest/region.html#Region.click

Feel free to share more about the issue and what the conversion is doing. This issue should be also replicable with TagUI. If so, you can consider joining the weekly TagUI Zoo Q&A call to troubleshoot further with my ex-colleague Ruth - aisingapore/TagUI#914

[kensoh]

Closing issue for now but please let me know if you run into any problem!