Help with Loop & Image recognition

Technical support and scripting issues

Moderators: Dorian (MJT support), JRL

Leve
Newbie
Posts: 8
Joined: Mon May 27, 2013 3:21 pm

Help with Loop & Image recognition

Hi,
Im currently using trial version of MS standard version.
I am not an expert in coding but I would like to build a script using MS for daily usage.

The condition of screen:

1. There are 2 buttons in 2 locations. location 'Z' and 'V'
2. The main condition that need to be found is 'picture1' on window screen
3. After clicking location 'Z' several times (random numbers), picture1 will disappear.

Basically what I want to do is something like this:
- The script try to find picture1 in certain area of screen.
- If picture1 is found, mouse move to location 'Z' and left click 'Z'
- Repeat the process until picture1 disappear.
- If picture1 not found, mouse move to location 'V' and left click 'V'

BUT during the repetitive process where picture1 is found and after clicking 'Z', sometimes the screen shows a whole different picture (call it picture2) at random occasion on top of picture1 which make picture1 hidden behind but the buttons still visible

- I would like the script to close this picture2 by clicking button 'Z' and continue to the process of finding picture1 until it disappear by itself (not because of picture2's appearances).

How should I make the loop conditions processes for above conditions?

Really appreciate the help

Last edited by Leve on Tue May 28, 2013 6:26 am, edited 1 time in total.


User avatar

CyberCitizen
Automation Wizard
Posts: 721
Joined: Sun Jun 20, 2004 7:06 am
Location: Adelaide, South Australia

Post by CyberCitizen » Tue May 28, 2013 2:35 am

Is there any chance you can provide screenshots so we can see what we are working with?

FIREFIGHTER


Leve
Newbie
Posts: 8
Joined: Mon May 27, 2013 3:21 pm

pictures

Post by Leve » Tue May 28, 2013 6:09 am

Unfortunately im not allowed to take the program's picture but it is something like this.

Image

Then sometimes picture2 appear. (to close it, press Z button)

Image

Is it good enough for pictures representatives ?

Last edited by Leve on Tue May 28, 2013 6:26 am, edited 1 time in total.


Leve
Newbie
Posts: 8
Joined: Mon May 27, 2013 3:21 pm

attempt

Post by Leve » Tue May 28, 2013 6:25 am

So I tried to attempt the logic for it.

This is what i done so far.

Code: Select all

                  FindImagePos.....picture1.bmp..... If>NumFound>0  Let>k=0  Repeat>k  Let>k=k+1   Mousemove>(button Z location)  LClick  Else  Mousemove>(button V location)  LClick   Let>k=NumFound   Until>k=NumFound EndIf                                  

Something like that works fine until picture2 appear and decide to click button V.
I would like the script to close picture2 and continue finding picture1 instead of quitting


User avatar

CyberCitizen
Automation Wizard
Posts: 721
Joined: Sun Jun 20, 2004 7:06 am
Location: Adelaide, South Australia

Post by CyberCitizen » Tue May 28, 2013 7:18 am

Do the buttons themselves stay stationary (eg a fixed position)?

Could you PM me a screenshot (not post publicly)?

FIREFIGHTER


Leve
Newbie
Posts: 8
Joined: Mon May 27, 2013 3:21 pm

Post by Leve » Tue May 28, 2013 7:34 am

Yes everything in fixed position like in the images above. nothing having random position.
I couldnt release actual program's screen since its against company's policy.
So this is the best I could do. Sorry


User avatar

CyberCitizen
Automation Wizard
Posts: 721
Joined: Sun Jun 20, 2004 7:06 am
Location: Adelaide, South Australia

Post by CyberCitizen » Tue May 28, 2013 7:57 am

Ok this is what I came up with, will need some tweaking for your purposes and maybe adjusting the wait times, not sure if your aware but searching the screen for images can be quite CPU intensive, so if you can limit it to a certain area then it doesn't have to search such a large space.

Image

Code: Select all

                  Label>Start //Find and Left Click Center of  FindImagePos>%BMP_DIR%\image_1.bmp,SCREEN,0.7,1,XArr,YArr,NumFound1,CCOEFF Wait>2 If>NumFound1>0   GoSub>Pic1Check EndIf  //Find and Left Click Center of  FindImagePos>%BMP_DIR%\image_2.bmp,SCREEN,0.7,1,XArr,YArr,NumFound2,CCOEFF Wait>2 If>NumFound2>0   GoSub>Pic2Check EndIf  If>{(%NumFound1%="0") AND (%NumFound2%="0")}   GoSub>NoPicsFound   Goto>Start Else   Goto>Start EndIf  SRT>Pic1Check   // Move Mouse To Button Z Location    // MouseMove>XArr_0,YArr_0   // LClick   MDL>I Sent A LClick Command On Button Z END>Pic1Check  SRT>Pic2Check   // Move Mouse To Button Z Location    // MouseMove>XArr_0,YArr_0   // LClick   MDL>I Sent A LClick Command On Button Z END>Pic2Check  SRT>NoPicsFound   // Move Mouse To Button V Location    // MouseMove>XArr_0,YArr_0   // LClick   MDL>I Sent A LClick Command On Button V END>NoPicsFound                

FIREFIGHTER


Leve
Newbie
Posts: 8
Joined: Mon May 27, 2013 3:21 pm

attempt

Post by Leve » Tue May 28, 2013 9:21 am

I managed to edit the script and running it.
Its working as intended but after the script clicking 'V' button, its still running and trying to click on 'V' button's location unless I stop it.

So I change Line18 to Goto>nextStep

and put Label>nextStep after Line42 to let the script continue doing something else

:lol:
I hope it was the right thing to do to let the script continue with something else or to let the script stop.

Thanks a lot for your help CyberCitizen.
Now I realize this sofware have so many potentials. quite happy with the result so far


User avatar

CyberCitizen
Automation Wizard
Posts: 721
Joined: Sun Jun 20, 2004 7:06 am
Location: Adelaide, South Australia

Post by CyberCitizen » Tue May 28, 2013 9:48 am

Hey,

No that is fine, I thought you wanted it to keep going after V was pressed, I assumed that pic 1 would pop back up and the process would continue.

Other option would be to remove line 18.

FIREFIGHTER


Leve
Newbie
Posts: 8
Joined: Mon May 27, 2013 3:21 pm

Post by Leve » Tue May 28, 2013 10:13 am

Well at beginning picture1 always popup even after hitting button 'Z'
but after a while (at random time) picture1 will disappear (not blocked by picture2) in which i want the script to press button 'V'.
Once 'V' button pressed, I would like the script to stop or continue with other processes(for future use).


armsys
Automation Wizard
Posts: 1108
Joined: Wed Dec 04, 2002 10:28 am
Location: Hong Kong

Post by armsys » Tue May 28, 2013 12:49 pm

CyberCitizen wrote://Find and Left Click Center of
FindImagePos>%BMP_DIR%\image_2.bmp,SCREEN,0.7,1,XArr,YArr,NumFound2,CCOEFF
Wait>2
If>NumFound2>0
GoSub>Pic2Check
EndIf

After completing FindImagePos>, it already generates a result-True & False. Why do your script waits 2 more seconds?


User avatar

CyberCitizen
Automation Wizard
Posts: 721
Joined: Sun Jun 20, 2004 7:06 am
Location: Adelaide, South Australia

Post by CyberCitizen » Wed May 29, 2013 12:12 am

armsys wrote:After completing FindImagePos>, it already generates a result-True & False. Why do your script waits 2 more seconds?

Because the script is to cycle though and keep clicking the Z Button until Picture 1 disappears, as the FindImagePos can be quite CPU intensive and if you didn't have the waits in there you will find the CPU usage for that app would go up. Had the issue with a similar loop script that I had, without the Wait in there it would be processing the same loop over and over much quicker than what was required.

http://www.mjtnet.com/usergroup/waitscr ... pu%20usage

FIREFIGHTER


armsys
Automation Wizard
Posts: 1108
Joined: Wed Dec 04, 2002 10:28 am
Location: Hong Kong

Post by armsys » Wed May 29, 2013 2:04 am

CyberCitizen wrote:Because the script is to cycle though and keep clicking the Z Button until Picture 1 disappears, as the FindImagePos can be quite CPU intensive and if you didn't have the waits in there you will find the CPU usage for that app would go up. Had the issue with a similar loop script that I had, without the Wait in there it would be processing the same loop over and over much quicker than what was required.

Hi CyberCitizen,
Thanks for your explanation.
For some odd reason, CCOEFF always takes much longer to complete.
I would prefer whenever possible:

Code: Select all

                  FindImagePos>%BMP_DIR%\image_2.bmp,SCREEN,0,1,XArr,YArr,NumFound2                

User avatar

CyberCitizen
Automation Wizard
Posts: 721
Joined: Sun Jun 20, 2004 7:06 am
Location: Adelaide, South Australia

Post by CyberCitizen » Wed May 29, 2013 2:24 am

Yes, that is actually something I should of mentioned, if its only going to be run on his machine & no one else would be using it on different OS's etc, then we could of just changed it to a simple find image position.

That being said I have supplied him some base code that he can now tweak to better suit his needs.

The reason the Correlation Coefficient algorithm takes longer is its not just searching for an exact match, it can find similarities in the images to best match them.

Version 14 introduces the use of a Correlation Coefficient algorithm. To use this set MatchAlgorithm to CCOEFF. This algorithm uses a template matching system attempting to find similarities in images rather than comparing pixels precisely. It is therefore able to find images even if they are not exactly the same and will therefore cope with differences in bit depth, different versions of images across different versions of Windows and different fonts, etc, making image recognition scripts more portable and less likely to fail should the environment change.

Each method has benefits and drawbacks. CCOEFF is more intelligent and more tolerant but is slower and will return only one match. EXACT is faster and can return multiple matches but is precise and therefore less portable and will not cope with changes.

FIREFIGHTER


Leve
Newbie
Posts: 8
Joined: Mon May 27, 2013 3:21 pm

Post by Leve » Wed May 29, 2013 4:37 am

Just happened to drop by out of curiosity.
Yes, Im only using the script on my machine, and nowhere else.
So is it better to remove the CCOEFF thing ?

Although the images that need to be founded are having gradient colors and various colors mixed in it. not a solid 1 color image.