Electron custom web browser with on screen keyboard

Question

I am trying to make a custom web browser inside an electron application. Using webview (because iframe is not loading some necessary web pages) I can load a web page. Then trying to write something into the web page´s input by clicking on the react-simple-keyboard which causes blur event, so input loses focus. I figured out, that this approach would not work directly, so via ipc communication I am trying to resend the key button value and then set it to the window with const {keyboard} = require("@nut-tree/nut-js"); keyboard.type(args.value);

In my input, above the webview tag, it works like a charm, but I am not able to type inside the webview.

Can anyone help me to solve this problem or does anyone know a perfect solution how to use other OSK in electron app or how to open native windows osk on input focus? Thank you in advance.

Answer 1

I'm not sure how you'd accomplish that with this library. But you can just use Window's default on-screen-keyboard to accomplish that. Here is a link to how enable it. windows support

You should also use a BrowserView instead of a Webview, as the Webview is not guaranteed to be present in future versions and it's API is unstable. The BrowserView doesn't work like an HTML element though and you should read the docs here.

But anyways, just use the system's default and you should be fine.

Also, if you're interested, I'm developing a web browser with Electron (in fact, I'm currently writing this using that browser) and as far as I can say, it's written pretty simply and anyone should understand most of it, so take a look if you're in trouble. But I am no expert and you shouldn't rely on my code as a standard of any kind, really.

Answer 2

Well, I might have just found an answer for you.

Firstly, as I mentioned, you should use a BrowserView instead of webView for your external content, and this time it is a requirement for this method to work. I would create a BrowserWindow with the controls at the top, then place a BrowserView to act as a "browser" and create another BrowserView at the bottom and load in the keyboard html file. And then, when a key is pressed on the virtual keyboard, you should send an ipc message to the main script with the information of what key was pressed(it should be done via a preload script for the OSK BrowserView). In the main script, once you recieve the ipc message (via ipcMain.on()) you should then send an input event to the BrowserView containing your external content. That's done by calling contents.sendInputEvent(Event), so it has to be a main script. Here is a link to contents.sendInputEvent(Event), BrowserView (link) and preload script as well as ipc communication (link).

As for invoking the keyboard once you click on the input element, you could probably do it with a preload script for your "browser's" BrowserView, if you can find how you can check whether the focused element is an input element or something like that, and call an ipc message to then hide or show the keyboard. (Hiding and shwoing the keyboard could be done by calling BrowserWindow.addBrowserView(BrowserView) or BrowserWindow.removeBrowserView(BrowserView). But you would have to search the documentation yourself for those methods as I can't write anymore right now. Documentation could anwser any of your questions if you search for it there.