JWinPointer.jar: multitouch and stylus pen input events on Microsoft Windows 10 for Java
JWinPointer.jar is a Java library for retrieving (and distinguishing between) mouse, multitouch, and stylus pen input events
with pressure, hover, and button information on Microsoft Windows 10, 64 bit.
As far as I know, it is the only single library for Java available that gives access to both multitouch and stylus events on Windows.
For ease of compilation, it has been packaged as a single .jar file
that contains the necessary .DLL files.
The screen shot above was created on a Microsoft Surface Pro 3 running Windows 10.
Each blue circle is either a mouse cursor (device==0),
stylus (device==1), or finger (device==2) with a unique id.
The grey text shows a scrolling log of events, useful for debugging
and understanding the sequence of events received.
The example code demonstrates how to integrate Swing widgets (like the buttons and menu bar visible above),
which still function normally
without interference from JWinPointer.
Below the window, we see the console displaying messages on standard output (stdout).
JWinPointer.jar was created in 2015 by Emery Monzerol,
a student at ETS,
and uses jni4net to connect Java to the native API in Windows.
Michael McGuffin created the example test .java program shown above.
Many thanks go out to Christophe Viau
who built the first FTIR multitouch tabletop surface at ETS
in 2010 using TUIO, which was the start of several multitouch development efforts at ETS.
How do I compile the example code?
Instructions for Eclipse:
- Download and install the latest 64-bit JDK (Java standard edition (SE) Development Kit) from Oracle's website
(at least Java 1.8)
- Download and install the latest 64-bit version of Eclipse
- Run Eclipse
- Use "File"/"New"/"Java Project" to create a new, empty, Java application
- Under the src directory of the new project, copy the .java file downloaded from the link above
- Under the root directory of the project, create a lib directory (i.e., at the same level as the src directory).
Copy the .jar file into that lib directory.
- Back in Eclipse, right-click on the name of the project in the Package Explorer, and select "Refresh".
Right click again on the name of the project, and select "Properties", which should open a dialog box.
Select "Java Build Path", and then the "Libraries" tab, and then hit the "Add External JARs"
button and navigate to the lib directory to select the .jar file.
- The code should now compile and run.
Are there any alternatives?
Alternative libraries and approaches you might consider for obtaining multitouch input events on Microsoft Windows:
- MT4j was a Java library for obtaining multitouch events.
However, the main website now seems defunct.
As far as I know, it was never available in 64 bit,
and did not support stylus events.
- You could write a TUIO client in Java (or another language)
and use a software bridge that translates native Microsoft multitouch events into TUIO events
that are then received by your client on the same local machine.
A few such bridges appear to be available.
Writing a TUIO client would have the advantage of being somewhat more portable to other platforms
(as long as there is some way of generating TUIO events on the target platform).
However, in the past when I tried using one of these bridges (W2TUIO.exe) on Microsoft Windows,
I found that it worked by creating a full-screen transparent window to capture the multitouch events --
this meant that my client was unable to receive any keyboard events or mouse events.
I don't know if TUIO has support for stylus events.
- Simple Multi-Touch Toolkit (SMT)
can be used as a library for Processing or as a standalone Java library.
According to its github page, it supports TUIO, Windows Touch, Leap Motion, SmartSDK,
mouse emulation, and works on Windows, Mac, and Linux.
- If you like writing Python code, try Kivy
(the successor to PyMT).
- If you like C/C++/C# programming, you could use the native API provided by Microsoft Windows.
Search for documentation on WM_TOUCH, Microsoft.Ink.DLL, RealTimeStylus.
- As far as I know, none of the above methods enable you to obtain simultaneous touch and stylus events
(i.e. where one or more fingers and stylus are touching the screen at the same time).
Typically, doing this will cause the touch events to be blocked and only allow stylus events through.
To obtain touch and stylus events simultaneously,
I know of two approaches, both of which only work with Wacom hardware like the Cintiq 24HD touch:
- If you like C programming, you could use the Wintab library provided by Wacom.
This will only work on Microsoft Windows.
- Another approach is to read the raw USB events from a Wacom device using a library like libusb,
which is available in multiple languages and on multiple platforms,
without using any Wacom drivers.
I did this and documented it in detail.
I was able to get this working on Linux using both C++ and Java, and on MS Windows 7 using Java,
but there was a problem on Windows 7 that resulted in a low rate of input events (only about 10 events/second on Windows 7
instead of more than 50 on Linux).
It also took me some time to reverse engineer the USB events,
and the code to interpret these would need to be debugged and updated with each new Wacom device that you want to use.
Cool things to do with multitouch and stylus input
- Examples using multitouch input
- Examples using stylus pen input
- Examples using both multitouch and stylus pen input
- Examples using both multitouch and stylus pen input simultaneously
(keywords for search engines: touch, multi-touch, pen pressure, stylus pressure)