Thursday, February 19, 2015

AutoHotKey

I want to setup some automatic regression tests of the GUI application that I help develop at work. I searched and found "Driving a Windows GUI program from a script" on stackoverflow which suggested AutoHotKey.

AutoHotKey is a mature, powerful, free packages with excellent documentation, but it is not designed for automatic regression testing. Here's an example using Notepad.

; Demonstration of Automated Test capabilities using Notepad.exe

#NoEnv                        ; ensure compatibility with future AutoHotkey releases
SendMode, Input               ; this mode has the best speed and reliability
SetWorkingDir, %A_ScriptDir%  ; ensure a consistent starting directory

; Launch Notepad
Run, Notepad.exe

; Use AutoIt3 Window Spy (which ships with AutoHotKey) to discover the window 
; ahk_class and the ClassNN of various controls. As long as you only run one instance 
; of the application you intend to auto-test, lookup by ahk_class is sufficient. 

; Wait until the window is loaded or your attempts to access its controls will fail. 
WinWaitActive, ahk_class Notepad

; Invoke the Page Setup menuItem because that window will have more controls
; and makes a better demo.
WinMenuSelectItem, ahk_class Notepad, , File, Page Setup...

; The ahk_class of the Page Setup window is #32770.

; Wait until the window is loaded or your attempts to access its controls will fail. 
WinWaitActive, ahk_class #32770

;--------------------

; We can learn things about controls 
ControlGet, EditBoxValue, Line, 1, Edit1, ahk_class #32770
MsgBox, Using Line Command: `nEdit-Box-Value = %EditBoxValue% `nReturn Code = %ErrorLevel%

; The text of an edit box is the same as the value accessed by the line command
ControlGetText, EditBoxText, Edit1, ahk_class #32770
MsgBox, Using Text Command: `nEdit-Box-Text = %EditBoxText% `nReturn Code = %ErrorLevel%

; We can get the Hwnd of controls so that we can reference them non-ambigously
ControlGet, EditBoxHwnd, Hwnd, , Edit1, ahk_class #32770
MsgBox, Getting Hwnd: `nEdit-Box-Hwnd = %EditBoxHwnd% `nReturn Code = %ErrorLevel%

; Once you have the Hwnd of a control, you can do useful things with it.
; Hwnd is unique accross all processes in a desktop, thus once you have the Hwnd
; of a control, you no longer need to specify the parent window.
ControlGet, EditBoxValue, Line, 1, , ahk_id %EditBoxHwnd%
MsgBox, Using Line Command with Hwnd: `nEdit-Box-Value = %EditBoxValue% `nReturn Code = %ErrorLevel%

; The text of an edit box is the same as the value accessed by the line command
ControlGetText, EditBoxText, , ahk_id %EditBoxHwnd%
MsgBox, Using Text Command with Hwnd: `nEdit-Box-Text = %EditBoxText% `nReturn Code = %ErrorLevel%

;--------------------

; We can access controls that contain other controls
ControlGet, PanelHwnd, Hwnd, , Button5, ahk_class #32770
MsgBox, Parent Controls: `nPanel-Hwnd = %PanelHwnd% `nReturn Code = %ErrorLevel%

; And learn about them
ControlGetText, PanelText, , ahk_id %PanelHwnd%
MsgBox, Parent Controls: `nPanel-Text = %PanelText% `nReturn Code = %ErrorLevel%

;--------------------

; You can get the Hwnd of a window so you can reference by non-ambigous Hwnd
WinGet, PageSetupHwnd, ID, ahk_class #32770
MsgBox, Window Hwnd = %PageSetupHwnd% `nReturn Code = %ErrorLevel%

; You can list all the children of a window, referencing the window by ahk_class.
WinGet, Children, ControlList, ahk_class #32770
MsgBox, Identified by Class: `nChildren-ClassNN-Names: `n%Children% `nReturn Code = %ErrorLevel%

; You can list all the children of a window, referencing the window by Hwnd.
WinGet, Children, ControlList, ahk_id %PageSetupHwnd%
MsgBox, Identified by Hwnd: `nChildren-ClassNN-Names: `n%Children% `nReturn Code = %ErrorLevel%

;--------------------

; You can't use the same command to list the children of a control
WinGet, Children, ControlList, ahk_id %PanelHwnd% 
MsgBox, Can't get children of a control: `n%Children% `nReturn Code = %ErrorLevel%

;--------------------

; In a complex GUI composed of frames, it is very likely that ClassNN names won't be unique.
; In this case you can distinguish them by position or by parent. Parent is probably the least
; brittle, since some parent will eventually have a unique ClassNN name.

;--------------------

; Some Win commands do operate on controls. This reveals that the ClassNN is an always-unique
; value that is always assigned at runtime. I had thought it was the variable name used in 
; the underlying program, but it really is just the class name with a run-time-unique postfix.
; This means that the only way to drive an auto-test that has any chance of not breaking in
; future versions of the software is to identify the controls by their text.

WinGetClass, ClassName, ahk_id %PageSetupHwnd%
MsgBox, Class-Name-Of-Window = %ClassName% `nReturn Code = %ErrorLevel%

WinGetClass, ClassName, ahk_id %PanelHwnd%
MsgBox, Class-Name-Of-Component = %ClassName% `nReturn Code = %ErrorLevel%

; For example, to find the Edit1 used above without the fore-knowledge that it is Edit1
; say versus Edit5, we could find it by knowing that it is within the "Margins (millimeters)"
; parent and the closest thing to the right of the "Left" label.

; For this we need to find controls bases on text/class, and filter based on position and
; parent relationships. While this is probably possible with AutoHotKey using the DllCall
; interface, it would be very difficult.

;--------------------

; You can close the applicaiton when you're done

WinClose ahk_id %PageSetupHwnd%
MsgBox, Closed Page Setup Window `nReturn Code = %ErrorLevel%

WinClose ahk_class Notepad
MsgBox, Closed Main Window `nReturn Code = %ErrorLevel%

The main problem here is that it has no built-in support to identify a control who's classNN might change, say in version2 of the software. A primary assumption of AutoHotKey is that you're hacking some else's already stable application, not building scripts that will be run against a changing GUI layout.

What I need is the ability to say: find me this unnamed control that is in this parent of this class with this name who is in that parent of that class of that name, then click it. I think that if I were to use AutoHotKey for this purpose, I'd have to write a custom DLL that provides that service and access it through the DllCall interface.

After a bit of searching, it seems like Sikuli or NunitForms would be good alternatives. The problem with Sikuli is that it is entirely based on image recognition, whereas I want the test script to embed hierarchical knowledge of controls within the application. The problem with NunitForms is that it doesn't operate on you compile application. It wants you to write a .NET application and instantiate the form to be tested within the test application.

Finally some good advice from stackoverflow:

I need to choose a Windows automation program. Which one do you recommend? AutoIt, AutoHotkey, or other?

AutoIt changed my life. It has became an invaluable tool in my work.

AutoIt has almost every feature AutoHotKey has and much more. COM-automation support, arrays and a pretty nice UDF (User Defined Functions) library.

That answer contains outdated information and currently is misleading. Currently AutoHotkey has COM-automation support, arrays, OOP, and lots of UDF (User Defined Functions) and libraries and many other improvements.

AutoHotkey has 3 major forks that blow away AutoIt. AutoHotkey_L, which has COM, Unicode support, Object-oriented like, Arrays, and more... AutoHotkeyCE which works on Windows mobile, PDAs, and smartphones. IronAHK, which is a .NET version of AutoHotkey that is 60% finished (is not developing any more).

If you are interesting in AutoHotkey under Windows (not Windows for mobile phones) always use AutoHotkey from http://ahkscript.org/ (current version, new official website)! AutoHotkey from autohotkey.com is outdated!

I use both depending on the situation. AutoHotkey is nice for quick keystroke macros and AutoIt has a much broader range of automation functionality and user-defined functions (UDFs) allow a range of useful things such as XML and database interaction. When automation requires a lot of GUI interaction I use AutoIt.

Note that in my tests above, I used autohotkey.com. Sure enough:

AHKScript.org is a new community consisting of the active AutoHotkey developer(s) as well as other enthusiasts. Unfortunately, the old autohotkey.com domain is not under the control of the developers and it continues to promote an outdated version of the software. New users are encouraged to migrate and participate at this new site.

Well, it turns out that the new AHK doesn't add any parent/child features, but I had a go at DllCall and it seems there's a lot you can do with just AHK and standard windows API. Here's an example with Notepad that doesn't actually work because it seems that the edit boxes in the "Margin" group-box aren't actually children of the group-box, but are just painted on top of it. In this demo, all the controls are listed in the EnumChildWindows call against the WindowHwnd and none are listed in the EnumChildWindows against the PanelHwnd.

; Demonstration of Automated Test capabilities using Notepad.exe

#NoEnv                        ; ensure compatibility with future AutoHotkey releases
#Warn                         ; Enable warnings to assist with detecting common errors.
SendMode, Input               ; this mode has the best speed and reliability
SetWorkingDir, %A_ScriptDir%  ; ensure a consistent starting directory

;--------------------

EnumChildWindowsCallback(Hwnd, lParam)
{
  WinGetTitle, Title, ahk_id %Hwnd%
  WinGetClass, Class, ahk_id %Hwnd%
  WinGetPos, X, Y, Width, Height, ahk_id %Hwnd%
  
  MsgBox, %Hwnd% `n%Title% `n%Class% `n%X% `n%Y% `n%Width% `n%Height%
  return true
}

; For performance call RegisterCallback once per callback.
; Fast-mode is okay because it will be called only from this thread.
EnumChildWindowsAddress := RegisterCallback("EnumChildWindowsCallback", "Fast")
MsgBox, EnumChildWindowsAddress = %EnumChildWindowsAddress% `nReturn Code = %ErrorLevel%

;--------------------

; Launch Notepad
Run, Notepad.exe

; Use AutoIt3 Window Spy (which ships with AutoHotKey) to discover the window 
; ahk_class and the ClassNN of various controls. As long as you only run one instance 
; of the application you intend to auto-test, lookup by ahk_class is sufficient. 

; Wait until the window is loaded or your attempts to access its controls will fail. 
WinWaitActive, ahk_class Notepad

; Invoke the Page Setup menuItem because that window will have more controls
; and makes a better demo.
WinMenuSelectItem, ahk_class Notepad, , File, Page Setup...

; The ahk_class of the Page Setup window is #32770.

; Wait until the window is loaded or your attempts to access its controls will fail. 
WinWaitActive, ahk_class #32770

;--------------------

; You can get the Hwnd of a window so you can reference by non-ambigous Hwnd
WinGet, PageSetupHwnd, ID, ahk_class #32770
MsgBox, Window-Hwnd = %PageSetupHwnd% `nReturn Code = %ErrorLevel%

; We can access controls that have unique caption text
ControlGet, PanelHwnd, Hwnd, , Margins (millimeters), ahk_class #32770
MsgBox, Panel-Hwnd = %PanelHwnd% `nReturn Code = %ErrorLevel%

; And learn about them
ControlGetText, PanelText, , ahk_id %PanelHwnd%
MsgBox, Panel-Text = %PanelText% `nReturn Code = %ErrorLevel%

;--------------------

; https://msdn.microsoft.com/en-us/library/windows/desktop/ms633494%28v=vs.85%29.aspx

MsgBox, Calling on PageSetupHwnd
Result := DllCall("EnumChildWindows", UInt, PageSetupHwnd, UInt, EnumChildWindowsAddress, UInt, 0)
MsgBox, Result = %Result% `nErrorLevel = %ErrorLevel%

MsgBox, Calling on PanelHwnd
Result := DllCall("EnumChildWindows", UInt, PanelHwnd, UInt, EnumChildWindowsAddress, UInt, 0)
MsgBox, Result = %Result% `nErrorLevel = %ErrorLevel%

;--------------------

; You can close the applicaiton when you're done

WinClose ahk_id %PageSetupHwnd%
MsgBox, Closed Page Setup Window `nReturn Code = %ErrorLevel%

WinClose ahk_class Notepad
MsgBox, Closed Main Window `nReturn Code = %ErrorLevel%

But I built my own Delphi test app that contains some buttons and then two instances of a frame that each contain a button. In this case, again the EnumChildWindows call against the WindowHwnd listed all the controls, even then non-direct children, but the EnumChildWindows against the PanelHwnd did list the button that is a child of that panel. Thus, you could get the Hwnd of a known component and then for example find the button with that component that has "Cancel" text and not have to worry about all the other "Cancel" buttons in that same window.

; Demonstration of Automated Test capabilities using Notepad.exe

#NoEnv                        ; ensure compatibility with future AutoHotkey releases
#Warn                         ; Enable warnings to assist with detecting common errors.
SendMode, Input               ; this mode has the best speed and reliability
SetWorkingDir, %A_ScriptDir%  ; ensure a consistent starting directory

;--------------------

EnumChildWindowsCallback(Hwnd, lParam)
{
  WinGetTitle, Title, ahk_id %Hwnd%
  WinGetClass, Class, ahk_id %Hwnd%
  WinGetPos, X, Y, Width, Height, ahk_id %Hwnd%
  
  MsgBox, %Hwnd% `n%Title% `n%Class% `n%X% `n%Y% `n%Width% `n%Height%
  return true
}

; For performance call RegisterCallback once per callback.
; Fast-mode is okay because it will be called only from this thread.
EnumChildWindowsAddress := RegisterCallback("EnumChildWindowsCallback", "Fast")
MsgBox, EnumChildWindowsAddress = %EnumChildWindowsAddress% `nReturn Code = %ErrorLevel%

;--------------------

; Launch Notepad
Run, C:\Code\Delphi\AutoHotKey\Win32\Release\AutoHotKey.exe

; Use AutoIt3 Window Spy (which ships with AutoHotKey) to discover the window 
; ahk_class and the ClassNN of various controls. As long as you only run one instance 
; of the application you intend to auto-test, lookup by ahk_class is sufficient. 

; Wait until the window is loaded or your attempts to access its controls will fail. 
WinWaitActive, ahk_class TForm1

;--------------------

; You can get the Hwnd of a window so you can reference by non-ambigous Hwnd
WinGet, WindowHwnd, ID, ahk_class TForm1
MsgBox, Window-Hwnd = %WindowHwnd% `nReturn Code = %ErrorLevel%

; You can get the Hwnd of a control
ControlGet, PanelHwnd, Hwnd, , TButtonFrame1, ahk_id %WindowHwnd%
MsgBox, TButtonFrame1-Hwnd = %PanelHwnd% `nReturn Code = %ErrorLevel%

;--------------------

; https://msdn.microsoft.com/en-us/library/windows/desktop/ms633494%28v=vs.85%29.aspx

MsgBox, Calling on WindowHwnd
Result := DllCall("EnumChildWindows", UInt, WindowHwnd, UInt, EnumChildWindowsAddress, UInt, 0)
MsgBox, Result = %Result% `nErrorLevel = %ErrorLevel%

MsgBox, Calling on PanelHwnd
Result := DllCall("EnumChildWindows", UInt, PanelHwnd, UInt, EnumChildWindowsAddress, UInt, 0)
MsgBox, Result = %Result% `nErrorLevel = %ErrorLevel%

;--------------------

; You can close the applicaiton when you're done

WinClose ahk_id %WindowHwnd%
MsgBox, Closed Page Setup Window `nReturn Code = %ErrorLevel%

Here's an example that uses DllCall to get information about the menus and submenus of the windows Calculator. You have to have the Calculator running for it to work. I tried using this instead of WinMenuSelectItem because in my application (which uses a Vcl.Menus TMainMenu with custom styling) WinMenuSelectItem fails to find my menus. Unfortunately, GetMenu also files to find my menu. Note that both work with a fresh project using a non-styled Vcl.Menus TMainMenu, although I didn't prove that the problem was the custom styling.

WinGet hWnd, ID, Calculator
msgbox, calculator hWnd = %hWnd% 

hMenu := DllCall("GetMenu", "UInt", hWnd)
msgbox, calculator hMenu = %hMenu% 

menuItemCount := DllCall("GetMenuItemCount", "UInt", hMenu)
Loop %menuItemCount%
{
  nPos := A_Index - 1
  msgbox nPos = %nPos%
  
  length := DllCall( "GetMenuString"
                   , "UInt", hMenu
                   , "UInt", nPos
                   , "UInt", 0       ; NULL
                   , "Int",  0       ; Get length
                   , "UInt", 0x0400) ; MF_BYPOSITION
  msgbox length1 = %length%
  
  bytelength := (length + 1) * 2
  msgbox bytelength = %bytelength%
  
  VarSetCapacity(lpString, bytelength)
  length := DllCall( "GetMenuString"
                   , "UInt", hMenu
                   , "UInt", nPos
                   , "Str",  lpString
                   , "Int",  bytelength
                   , "UInt", 0x0400)
  msgbox length2 = %length%
                   
  id := DllCall("GetMenuItemID", "UInt", hMenu, "Int", nPos)
  msgbox hMenu = %hMenu% `nid = %id% `nstring = %lpString%
  
  if (id = -1)
  {
    hSubMenu := DllCall("GetSubMenu", "Uint", hMenu, "int", nPos)
    msgbox, hSubMenu = %hSubMenu% 
  }
}
{ "loggedin": false, "owner": false, "avatar": "", "render": "nothing", "trackingID": "UA-36983794-1", "description": "", "page": { "blogIds": [ 583 ] }, "domain": "holtstrom.com", "base": "\/michael", "url": "https:\/\/holtstrom.com\/michael\/", "frameworkFiles": "https:\/\/holtstrom.com\/michael\/_framework\/_files.4\/", "commonFiles": "https:\/\/holtstrom.com\/michael\/_common\/_files.3\/", "mediaFiles": "https:\/\/holtstrom.com\/michael\/media\/_files.3\/", "tmdbUrl": "http:\/\/www.themoviedb.org\/", "tmdbPoster": "http:\/\/image.tmdb.org\/t\/p\/w342" }