After closing a modal JDialog, sometimes main window doesn't receive focus until another modal dialog is shown

Question

I apologise ahead of time of lack of reproducable example, the app i have is very big, and the problem seems to be related to weird combination of controls and focus switches, which i haven't been able to reproduce with a shorter program.

Basically, i have an text editor application (in java8 on windows 7) with a JFrame and a JTabbedPane with each tab containing a JSplitPane of text area and another tab displaying some information. I have implemented a tab switcher system like in Eclipse, Ctrl-Tab shows a list of editors in a modal JDialog. This is implemented as a KeyListener which keeps track of various keys pressed. The code looks something like this:

private TabsDialog td = new TabsDialog(mainFrame, true);

void onKeyPressCtrlTab()
{
  td.setVisible(true); //display tab dialog
  //After closing dialog, select correct tab
  TabContent tc = td.getSelectedTab();
  switchToTab(tc);
}

void onKeyRelease()
{
    dispatchDialog();
}

void dispatchDialog()
{
  if (td.isVisible())
  {
  td.setVisible(false);
  }
}

The problem is, if i invoke onKeyPressCtrlTab and onKeyRelease in quick succession by pressing and releasing Ctrl-Tab, the focus system of the main application stops working, i can click on the main window and select text with the mouse, but the caret in text components isn't being shown at all. I can't type any text into any of the text components either. Also, all requestFocusInWindow calls are failing.

The problem is reproducable every time, but sometimes it takes 5 of Ctrl-Tabs, sometimes longer.

I've traced the problem to Component#requestFocusHelper, the following call always fail after the problem appears:

final boolean requestFocusHelper (boolean temporary,
                                     boolean focusedWindowChangeAllowed,
                                     CausedFocusEvent.Cause cause)
...
        boolean success = peer.requestFocus
            (this, temporary, focusedWindowChangeAllowed, time, cause); //fails

It seems that calling a new modal dialog (for example JOptionPane#showConfirmDialog) restores the focus system, but this is not a great solution.

I have spend two days on this issue, but haven't been able to find any good solution. It hasn't helped to dispose and re-create the TabsDialog on every call. I suspect there's something wrong with how the dialogs are cleaned up / enabled in Windows.

I think i'm doing all work on AWT thread so i doubt that's the issue. There are "some" things going on when switching tabs, i have a splitpane which is set / restored since each tab remembers the position, but still, nothing that should affect the focus system in my opinion. I have a uncatched exception in thread filter, so no exceptions are supressed as far as i can see.

Appreciate any thoughts on the problem, and a potential solution (even workaround that restores the focus would be great)

Here's a demonstration of how it's supposed to work:

https://gist.github.com/siggemannen/4affdff4b1892a15e481c626a190efab

Answer 1

A pure Swing solution would be to use Key Bindings, not a 3rd party tool to handle key events.

As I understand your requirement, here is my simple implementation using Key Bindings:

import javax.swing.*;
import javax.swing.border.*;
import java.awt.*;
import java.awt.event.*;
import javax.swing.event.*;
import javax.swing.plaf.*;
import java.util.*;

public class TabbedPaneDialog extends JPanel
{
    JTabbedPane tabbedPane;

    public TabbedPaneDialog()
    {
        setLayout( new BorderLayout() );

        tabbedPane = new JTabbedPane();
        add(tabbedPane);

        newTab( "File 1" );
        newTab( "File 2" );
        newTab( "File 3" );
        newTab( "File 4" );
        newTab( "File 5" );

        // remove Control+Tab as a Tab key

        Set newForwardKeys = new HashSet();
        newForwardKeys.add( KeyStroke.getKeyStroke("TAB") );
        tabbedPane.setFocusTraversalKeys(KeyboardFocusManager.FORWARD_TRAVERSAL_KEYS, newForwardKeys);

        // Use Control+Tab to display dialog

        KeyStroke controlTab = KeyStroke.getKeyStroke("control TAB");
        InputMap im = tabbedPane.getInputMap(JTabbedPane.WHEN_ANCESTOR_OF_FOCUSED_COMPONENT);
        im.put(controlTab, "showDialog");
        tabbedPane.getActionMap().put("showDialog", new DialogAction("Show Dialog", tabbedPane));

    }

    private void newTab(String text)
    {
        JTextArea textArea = new JTextArea(10, 30);
        textArea.setText(text);
        JScrollPane scrollPane = new JScrollPane( textArea );
        tabbedPane.add( scrollPane, text );

        // remove Control+Tab as a Tab key

        Set newForwardKeys = new HashSet();
        newForwardKeys.add( KeyStroke.getKeyStroke("TAB") );
        textArea.setFocusTraversalKeys(KeyboardFocusManager.FORWARD_TRAVERSAL_KEYS, newForwardKeys);

        // Use Control+Tab to display dialog

        KeyStroke controlTab = KeyStroke.getKeyStroke("control TAB");
        InputMap im = textArea.getInputMap(JTabbedPane.WHEN_ANCESTOR_OF_FOCUSED_COMPONENT);
        im.put(controlTab, "showDialog");
        textArea.getActionMap().put("showDialog", new DialogAction("Show Dialog", tabbedPane));
    }

    class DialogAction extends AbstractAction
    {
        private JTabbedPane tabbedPane;

        public DialogAction(String name, JTabbedPane tabbedPane)
        {
            putValue( Action.NAME, name );
            this.tabbedPane = tabbedPane;
        }

        public void actionPerformed(ActionEvent e)
        {
            SelectionDialog dialog = new SelectionDialog(tabbedPane);
        }
    }

    class SelectionDialog
    {
        private JTabbedPane tabbedPane;

        public SelectionDialog(JTabbedPane tabbedPane)
        {
            this.tabbedPane = tabbedPane;

            DefaultListModel<String> model = new DefaultListModel<>();

            for (int i = 0; i < tabbedPane.getTabCount(); i++)
            {
                model.addElement( tabbedPane.getTitleAt(i) );
            }

            JList<String> list = new JList<>(model);
            list.setSelectedIndex( tabbedPane.getSelectedIndex() );

            Set newForwardKeys = new HashSet();
            newForwardKeys.add( KeyStroke.getKeyStroke("TAB") );
            list.setFocusTraversalKeys(KeyboardFocusManager.FORWARD_TRAVERSAL_KEYS, newForwardKeys);

            KeyStroke controlTab = KeyStroke.getKeyStroke("control TAB");
            InputMap im = list.getInputMap(JList.WHEN_FOCUSED);
            im.put(controlTab, "nextTab");
            list.getActionMap().put("nextTab", new NextTabAction("Next Tab", list));

            KeyStroke releasedControlTab = KeyStroke.getKeyStroke("released CONTROL");
            im = list.getInputMap(JList.WHEN_FOCUSED);
            im.put(releasedControlTab, "closeDialog");
            list.getActionMap().put("closeDialog", new CloseDialogAction("Close Dialog", list, tabbedPane));

            Window window = SwingUtilities.windowForComponent(tabbedPane);
            JDialog dialog = new JDialog(window);
            dialog.add(new JScrollPane(list));
            dialog.pack();
            dialog.setLocationRelativeTo( tabbedPane );
            dialog.setVisible(true);
        }

        class NextTabAction extends AbstractAction
        {
            private JList list;

            public NextTabAction(String name, JList list)
            {
                putValue( Action.NAME, name );
                this.list = list;
            }

            public void actionPerformed(ActionEvent e)
            {
                int selected = list.getSelectedIndex();
                selected++;

                if (selected == list.getModel().getSize())
                    selected = 0;

                list.setSelectedIndex( selected );
            }
        }

        class CloseDialogAction extends AbstractAction
        {
            private JList list;
            private JTabbedPane tabbedPane;

            public CloseDialogAction(String name, JList list, JTabbedPane tabbedPane)
            {
                putValue( Action.NAME, name );
                this.list = list;
                this.tabbedPane = tabbedPane;
            }

            public void actionPerformed(ActionEvent e)
            {
                int selected = list.getSelectedIndex();
                tabbedPane.setSelectedIndex( selected );

                Window window = SwingUtilities.windowForComponent(list);
                window.setVisible( false );

                JScrollPane scrollPane = (JScrollPane)tabbedPane.getComponentAt( selected );
                JTextArea textArea = (JTextArea)scrollPane.getViewport().getView();
                textArea.requestFocusInWindow();
            }
        }
    }

    public static void main(String args[])
    {
        JFrame frame = new JFrame("TabbedPane Dialog");
        frame.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );
        frame.add(new TabbedPaneDialog());
        frame.pack();
        frame.setVisible(true);
    }
}