AppleScript -- Menu Item in the Menu Button popup is not selectable while using keystroke return or (key code 36) command

Question

I have gone through stack overflow, this might seem like a duplicate question but this is more like an extended question to the existing issue!

I am facing a very strange issue, in my application we have customised the pop up button (Inheriting from NSPopupButton). When I use Apple Script to access the pop up item, the script is not able to select any value from the popup list.

Let me be more clear!

For eg. Imagine the Popup button has "Car, Bike, Cycle, Bus" as its items. Now if I use AppleScript to select Bike and then click enter, the value is not getting reflected. But manually if I select "Bike" it will reflect in the application. This is the script that I wrote.

click menu button "Car" of window 1 of application process "My Sample Application" of application "System Events"
        key code 125 //I am using these to navigate down the list
        delay 1
        key code 125 //I am using these to navigate down the list
        delay 1
        key code 125 //I am using these to navigate down the list
        delay 3
        key code 36 //This line simulated ENTER/Return key (or NOT?)

I have also tried running this command in Apple Script

--set xxx to value of every button of menu button "Car" of window 1 of application process "LG Calibration Studio" of application "System Events"

This command was suppose to show me all the items present in the Popup, but unfortunately it is returning an empty array(or Dictionary). Even though the pop up is populated with values I am not sure why this is returning empty.

I am kind of new to Apple Script and do not have much knowledge in this domain. Any help is appreciated. Thanks in advance :)

Answer 1

When using UI scripting, the relationships between ui objects is not always clear. I'll use Apple's Pages (as it's freely available — I use an untouched 'Essay' document) as an example. Here are two suggestions to explore. If they make sense, you can try similar actions with your own app.

UI Elements command

In a new script…

tell application "Pages"
    -- activate -- Not necessary until you want to act upon the window
    tell application "System Events" to tell front window of application process "Pages"
        UI elements -- generate list of window's ui elements
        properties of splitter group 1 -- the first element in the list (branch 1)
        UI elements of splitter group 1 -- generate list of first branch
        -- entire contents
    end tell
end tell

Partial result of command 1

{splitter group 1 of window "Untitled" of application process "Pages" of application "System Events", button 1 of window "Untitled" of application process "Pages" of application "System Events", button 2 of window "Untitled" of application process "Pages" of application "System Events"}

Partial result of command 2

{minimum value:missing value, orientation:missing value, position:{40, 22}}

Partial result of command 3

{scroll area 1 of splitter group 1 of window "Untitled" of application process "Pages" of application "System Events", static text "0 Comments & Changes" of splitter group 1 of window "Untitled" of application process "Pages" of application "System Events", button "Comment" of splitter group 1 of window "Untitled" of application process "Pages" of application "System Events"}

Using ui elements you can work your way through the myriad interface objects in your window. You can also ask for specific properties, e.g. name of every pop up button. Finally, you can get a list of all ui elements with entire contents.

Accessibility Inspector

Using the application Accessibility Inspector, you can inspect the window and discover its elements. This is located inside XCode's app bundle and is also accessible through the XCode > Open Developer Tool menu.

It changes with the version of XCode so there isn't any point in getting too detailed (I'm on Xcode 9) but once it is launched, click on the 'All Processes' drop down button at the top left and it should list the open apps; select Pages from this list.

A couple of things to note: At the bottom is the hierarchy of elements which can help in determining which object to act upon. Each line's bracketed text references the class of object but the text isn't always literal (e.g. your script should use 'window' not 'standard window').

If you use the 'pointer' and then in Pages, click on the empty area at the bottom of the Format inspector, the hierarchy will show a long list of items under (scroll area). Also, there are various ways to interact with an interface object, of which 'click' is but one.

If I insert this code into the script and then select the body text, the script will format the selection as italics (which matches the hierarchy entry 'Regular (pop up button)', with 'Regular' being the selection's existing font format.

tell pop up button 2 of scroll area 2 of splitter group 1 of ¬
    window "Untitled" of application process "Pages" of ¬
    application "System Events" to perform action "AXPress"
delay 0.5
keystroke "i"
key code 36

Hopefully, utilizing these two approaches may yield some useful results before the onset of ui scripting headache.