Building Project Panama’s jextract tool by yourself

Absorb what is useful, discard what is useless and add what is specifically your own. -- Bruce Lee

Do you want to build Project Panama's Jextract tool by yourself? I can show you how!

If you've seen any of my past articles on Project Panama I've mentioned a really convenient tool called jextract that can generate Java binding code that represents native functions or variables (symbols) from C libraries.

This alleviates the developer's need of creating binding code by hand. By passing in a C header file (.h extension) jextract can generate source code or compiled Java classes.

Did you know jextract will not be included in OpenJDK releases?

For those who are not familiar with the state of Java's Project Panama & the jextract tool let me explain. As of this writing the Java enhancement proposal JEP 424 will be making the foreign function and memory access APIs available as preview features in JDK 19-ea (early access) release and will not include the jextract tool.

Note: To enable preview features and native access do the following when running a Java program.

java --enable-preview --source 19 --enable-native-access=ALL-UNNAMED MyJavaApp.java

By enabling and using JDK 19's preview features you can begin to kick the tires and provide feedback to the engineers and community before it becomes final in the GA release of Java.

Prior to JEP 424 the early access builds of the OpenJDK used to contain the jextract tool as part of the JDK.

However, the decision was made to have the jextract tool become its own project over at GitHub and will not be included in builds of the OpenJDK.

Having said this, you'll want to be able to build the jextract tool yourself. Of course you can wait till a build is available, but why wait? In this article I will walk you through the process on how to build jextract yourself.

Assumptions

This article assumes you know what is Java's Foreign Function and Memory Access APIs (APIs from Project Panama) and basic knowledge of the following.

• Bash commands
• Git is installed
• Git commands

Requirement

• JDK 19 EA - The early access build of JDK 19 (Preview Release)
• jextract - A tool to generate binding code to allow developers to easily access native symbols. The jextract tool is now on GitHub. The master branch is a in synch with the latest OpenJDK version. If you would like to obtain a past build for a version check jdk<version>. At this time a branch jdk18 is a build.
• LLVM - A set of compiler tools to optimize and statically compile various languages into native machine code.
• Gradle (optional) - The Gradle build tool to build the jextract project.
• Git

Installing Software

First on the list of required software is JDK 19 EA. If you are a fan of using SDKMan then do the following:

sdk install java 19.ea.25-open

Note: If you have installed the JDK using SDKMan than you can skip to the jextract at GitHub section.

If you want to do install the JDK the old fashioned way follow the instructions below:

Head over to the early access release of JDK 19 site to download the latest build for your operating system as shown below.

After downloading JDK 19 decompress the file into a directory and set the JAVA_HOME and PATH environment variables as follows:

# MacOS 
$ export JAVA_HOME=<untarred_dir>/jdk-19.jdk/Contents/Home
$ export PATH=$JAVA_HOME/bin:$PATH

# Linux
$ export JAVA_HOME=<untarred_dir>/jdk-19
$ export PATH=$JAVA_HOME/bin:$PATH

# Windows
C:\> set JAVA_HOME=<unzipped_dir>\jdk-19
C:\> set PATH=%JAVA_HOME%\bin;%PATH%

Of course if you choose to make environment variables persistent you'll need to add them to the .bashrc or .bash_profile file of your Linux or MacOS environment. On the Windows operating system you will want to add or update environment variables in the control panel's System Properties -> Environment Variables.

jextract at GitHub

After setting up JDK 19 you can fork / clone the GitHub project jextract with the following command:

git clone [email protected]:openjdk/jextract.git
cd jextract

Note: It is preferable that you fork the project and then clone your fork of the project. That way you can provide pull requests whenever you find a bug to fix or a proposed enhancement to the project. But if you just want to build from master branch or a previous branch you can simply clone jextract's main repo as shown below:

Assuming your still in the directory of the repository you've just cloned (./jextract) next you'll download and install LLVM before building the jextract project.

LLVM

Next, you'll need to download LLVM (version 13.0.0) for your particular operating system and then decompress it into a directory. Afterwards, you'll need to execute the following command to build the jextract tool.

sh ./gradlew -Pjdk19_home=<jdk19_home_dir> -Pllvm_home=<libclang_dir> clean verify

Note: Substitute the following place holders <jdk19_home_dir> and <libclang_dir> with the JAVA_HOME's directory and LLVM's installed directory (clang+llvm-13.0.0-*) respectively. An example on my MacOS terminal is shown below.

sh ./gradlew \
   -Pjdk19_home=/Users/cdea/sdks/jdk-19.jdk/Contents/Home/ \
   -Pllvm_home=/Users/cdea/sdks/clang+llvm-13.0.0-x86_64-apple-darwin/ \
   clean verify

You're almost done! After building the tool the process will create a new JDK SDK image that will be used instead of your newly downloaded JDK 19. By passing in the -Pjdk19_home it will create and generate a copy of the JDK 19 and jextract binaries located in the following directories:

build/jextract/bin
build/jextract/lib

The JDK with jextract build directory should look like the following:

Re-setting the JAVA_HOME and PATH environment variables

Before we can use the jextract tool you'll need to change the previous JAVA_HOME and PATH to now point to the directory build/jextract and build/jextract/bin respectively.

Great, now that you have a fresh JDK containing the jextract, let's generate some binding code!

Generate Panama bindings

Another requirement of jextract is to have access to the C libraries. Using jextract to generate binding code on the various operating systems you will need to install C libraries and headers files. The following are instructions to install C libraries for the respective operating systems.

MacOS

In order to obtain C libraries and header files on a MacOS operating system you'll need Xcode to be installed. If you don't have Xcode install do the following:

xcode-select —install

Linux

In the case of Linux you'll need gcc's compiler and libraries. To install enter the following commands:

# Ubuntu
sudo apt update
sudo apt install build-essential

# CentOS
sudo yum groupinstall 'Development Tools'

Windows

When developing native C/C++ libraries you will download and decompress MinGW from  https://sourceforge.net/projects/mingw-w64/. After downloaded you can unzipped MinGW into the C: drive's root directory.

Now that you have all the required C libraries lets use jextract against the stdio.h file to generate binding code.

Use jextract against stdio.h

Now that you have all your C libraries and header files in place you can target the stdio.h file to generate binding code. The commands below will generate source code that will be outputted in the src directory. Use the appropriate commands based on your operating system.

# MacOS
export C_INCLUDE_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
jextract --source --output src -t org.unix -I $C_INCLUDE_PATH $C_INCLUDE_PATH/stdio.h

# Linux
export C_INCLUDE_PATH=/usr/include/
jextract --source --output src -t org.unix -I $C_INCLUDE_PATH $C_INCLUDE_PATH/stdio.h

# Windows
set C_INCLUDE_PATH=C:\MinGW\include
jextract.exe --source --output src -t org.unix -I %C_INCLUDE_PATH% %C_INCLUDE_PATH%\stdio.h

Using jextract to generate binding code for the Windows platform. This assumes you've installed MinGW in the C: directory.

# MacOS
export C_INCLUDE_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
jextract --output classes -t org.unix -I $C_INCLUDE_PATH $C_INCLUDE_PATH/stdio.h

# Linux
export C_INCLUDE_PATH=/usr/include/
jextract --output classes -t org.unix -I $C_INCLUDE_PATH $C_INCLUDE_PATH/stdio.h

rem Generate Clib classes
set C_INCLUDE_PATH=C:\MinGW\include
jextract.exe --output classes -t org.unix -I %C_INCLUDE_PATH% %C_INCLUDE_PATH%\stdio.h

Now that you've generated Java source code or classes they can be conveniently used in your Java applications.

Conclusion

You've now learned that the jextract tool has been separated into its own project and is no longer part of the OpenJDK build distributions.

To build the jextract tool, you learned how to install required software such as a C compiler and libraries.

Once required software is installed, you use the gradle build tool to build a JDK distribution containing JDK 19-ea along with the jextract tool located in the build/jextract/bin directory.

Lastly, you are now able to use jextract to generate binding code to invoke stdio.h C functions.

There you have it! You're able to build jextract by yourself.

• Developer Tools
• JEPs
• Performance
• Project Panama
• Tutorials

Carl Dea

Author

Carl Dea is a Senior Developer Advocate at Azul. He has authored Java books and has been developing software for 20+ years with many clients, from Fortune 500 companies to nonprofit organizations. He has written software ranging from mission-critical applications to e-commerce applications. Carl has been using Java since the very beginning (when Applets were cool) and is a JavaFX enthusiast (fanboy) dating back to when it used to be called F3/JavaFX script. He greatly loves sharing and advocating Java based technologies.