Friends of OpenJDK Today

Building Project Panama’s jextract tool by yourself

June 09, 2022

Author(s)

  • Carl Dea

    Carl Dea is a Senior Developer Advocate at Azul. He has authored Java books and has been developing software for 20+ years with many clients, from Fortune 500 companies to ... Learn more

Absorb what is useful, discard what is useless and add what is specifically your own. -- Bruce Lee

Do you want to build Project Panama's Jextract tool by yourself? I can show you how!

If you've seen any of my past articles on Project Panama I've mentioned a really convenient tool called jextract that can generate Java binding code that represents native functions or variables (symbols) from C libraries.

This alleviates the developer's need of creating binding code by hand. By passing in a C header file (.h extension) jextract can generate source code or compiled Java classes.

Did you know jextract will not be included in OpenJDK releases?

For those who are not familiar with the state of Java's Project Panama & the jextract tool let me explain. As of this writing the Java enhancement proposal JEP 424 will be making the foreign function and memory access APIs available as preview features in JDK 19-ea (early access) release and will not include the jextract tool.

Note: To enable preview features and native access do the following when running a Java program.

java --enable-preview --source 19 --enable-native-access=ALL-UNNAMED MyJavaApp.java

By enabling and using JDK 19's preview features you can begin to kick the tires and provide feedback to the engineers and community before it becomes final in the GA release of Java.

Prior to JEP 424 the early access builds of the OpenJDK used to contain the jextract tool as part of the JDK.

However, the decision was made to have the jextract tool become its own project over at GitHub and will not be included in builds of the OpenJDK.

Having said this, you'll want to be able to build the jextract tool yourself. Of course you can wait till a build is available, but why wait? In this article I will walk you through the process on how to build jextract yourself.

Assumptions

This article assumes you know what is Java's Foreign Function and Memory Access APIs (APIs from Project Panama) and basic knowledge of the following.

  • Bash commands
  • Git is installed
  • Git commands

Requirement

  • JDK 19 EA - The early access build of JDK 19 (Preview Release)
  • jextract - A tool to generate binding code to allow developers to easily access native symbols. The jextract tool is now on GitHub. The master branch is a in synch with the latest OpenJDK version. If you would like to obtain a past build for a version check jdk<version>. At this time a branch jdk18 is a build.
  • LLVM - A set of compiler tools to optimize and statically compile various languages into native machine code.
  • Gradle (optional) - The Gradle build tool to build the jextract project.
  • Git

Installing Software

First on the list of required software is JDK 19 EA. If you are a fan of using SDKMan then do the following:

sdk install java 19.ea.25-open

Note: If you have installed the JDK using SDKMan than you can skip to the jextract at GitHub section.

If you want to do install the JDK the old fashioned way follow the instructions below:

Head over to the early access release of JDK 19 site to download the latest build for your operating system as shown below.

OpenJDK JDK 19 Early-Access Builds Download Page

After downloading JDK 19 decompress the file into a directory and set the JAVA_HOME and PATH environment variables as follows:

# MacOS 
$ export JAVA_HOME=<untarred_dir>/jdk-19.jdk/Contents/Home
$ export PATH=$JAVA_HOME/bin:$PATH

# Linux
$ export JAVA_HOME=<untarred_dir>/jdk-19
$ export PATH=$JAVA_HOME/bin:$PATH

# Windows
C:\> set JAVA_HOME=<unzipped_dir>\jdk-19
C:\> set PATH=%JAVA_HOME%\bin;%PATH%

Of course if you choose to make environment variables persistent you'll need to add them to the .bashrc or .bash_profile file of your Linux or MacOS environment. On the Windows operating system you will want to add or update environment variables in the control panel's System Properties -> Environment Variables.

jextract at GitHub

After setting up JDK 19 you can fork / clone the GitHub project jextract with the following command:

git clone [email protected]:openjdk/jextract.git
cd jextract

Note: It is preferable that you fork the project and then clone your fork of the project. That way you can provide pull requests whenever you find a bug to fix or a proposed enhancement to the project. But if you just want to build from master branch or a previous branch you can simply clone jextract's main repo as shown below:

Assuming your still in the directory of the repository you've just cloned (./jextract) next you'll download and install LLVM before building the jextract project.

LLVM

Next, you'll need to download LLVM (version 13.0.0) for your particular operating system and then decompress it into a directory. Afterwards, you'll need to execute the following command to build the jextract tool.

sh ./gradlew -Pjdk19_home=<jdk19_home_dir> -Pllvm_home=<libclang_dir> clean verify

Note: Substitute the following place holders <jdk19_home_dir> and <libclang_dir> with the JAVA_HOME's directory and LLVM's installed directory (clang+llvm-13.0.0-*) respectively. An example on my MacOS terminal is shown below.

sh ./gradlew \
   -Pjdk19_home=/Users/cdea/sdks/jdk-19.jdk/Contents/Home/ \
   -Pllvm_home=/Users/cdea/sdks/clang+llvm-13.0.0-x86_64-apple-darwin/ \
   clean verify

You're almost done! After building the tool the process will create a new JDK SDK image that will be used instead of your newly downloaded JDK 19. By passing in the -Pjdk19_home it will create and generate a copy of the JDK 19 and jextract binaries located in the following directories:

build/jextract/bin
build/jextract/lib

The JDK with jextract build directory should look like the following:

Re-setting the JAVA_HOME and PATH environment variables

Before we can use the jextract tool you'll need to change the previous JAVA_HOME and PATH to now point to the directory build/jextract and build/jextract/bin respectively.

Great, now that you have a fresh JDK containing the jextract, let's generate some binding code!

Generate Panama bindings

Another requirement of jextract is to have access to the C libraries. Using jextract to generate binding code on the various operating systems you will need to install C libraries and headers files. The following are instructions to install C libraries for the respective operating systems.

MacOS

In order to obtain C libraries and header files on a MacOS operating system you'll need Xcode to be installed. If you don't have Xcode install do the following:

xcode-select —install

Linux

In the case of Linux you'll need gcc's compiler and libraries. To install enter the following commands:

# Ubuntu
sudo apt update
sudo apt install build-essential

# CentOS
sudo yum groupinstall 'Development Tools'

Windows

When developing native C/C++ libraries you will download and decompress MinGW from  https://sourceforge.net/projects/mingw-w64/. After downloaded you can unzipped MinGW into the C: drive's root directory.

Now that you have all the required C libraries lets use jextract against the stdio.h file to generate binding code.

Use jextract against stdio.h

Now that you have all your C libraries and header files in place you can target the stdio.h file to generate binding code. The commands below will generate source code that will be outputted in the src directory. Use the appropriate commands based on your operating system.

# MacOS
export C_INCLUDE_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
jextract --source --output src -t org.unix -I $C_INCLUDE_PATH $C_INCLUDE_PATH/stdio.h

# Linux
export C_INCLUDE_PATH=/usr/include/
jextract --source --output src -t org.unix -I $C_INCLUDE_PATH $C_INCLUDE_PATH/stdio.h

# Windows
set C_INCLUDE_PATH=C:\MinGW\include
jextract.exe --source --output src -t org.unix -I %C_INCLUDE_PATH% %C_INCLUDE_PATH%\stdio.h

Using jextract to generate binding code for the Windows platform. This assumes you've installed MinGW in the C: directory.

# MacOS
export C_INCLUDE_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
jextract --output classes -t org.unix -I $C_INCLUDE_PATH $C_INCLUDE_PATH/stdio.h

# Linux
export C_INCLUDE_PATH=/usr/include/
jextract --output classes -t org.unix -I $C_INCLUDE_PATH $C_INCLUDE_PATH/stdio.h

rem Generate Clib classes
set C_INCLUDE_PATH=C:\MinGW\include
jextract.exe --output classes -t org.unix -I %C_INCLUDE_PATH% %C_INCLUDE_PATH%\stdio.h

Now that you've generated Java source code or classes they can be conveniently used in your Java applications.

Conclusion

You've now learned that the jextract tool has been separated into its own project and is no longer part of the OpenJDK build distributions.

To build the jextract tool, you learned how to install required software such as a C compiler and libraries.

Once required software is installed, you use the gradle build tool to build a JDK distribution containing JDK 19-ea along with the jextract tool located in the build/jextract/bin directory.

Lastly, you are now able to use jextract to generate binding code to invoke stdio.h C functions.

There you have it! You're able to build jextract by yourself.

Topics:

Related Articles

View All

Author(s)

  • Carl Dea

    Carl Dea is a Senior Developer Advocate at Azul. He has authored Java books and has been developing software for 20+ years with many clients, from Fortune 500 companies to ... Learn more

Comments (5)

Your email address will not be published. Required fields are marked *

Highlight your code snippets using [code lang="language name"] shortcode. Just insert your code between opening and closing tag: [code lang="java"] code [/code]. Or specify another language.

Save my name, email, and website in this browser for the next time I comment.

Joao

You say that if one has installed the JDK via sdkman, you don’t need to do jextract at github.
But then you don’t have the folder and you can’t run gradle commands like clean and verify.

Carl Dea

Joao,
When I say ‘skip to the jextract at GitHub section’ it means to go (jump) to the jextract at GitHub section. Setting up the JAVA_HOME instructions are not needed if you use SDKMan.

I hope that makes sense. American English can be confusing.

-Carl

Error while building Jextract from Project Panama - TechTalk7

[…] tried following this guide step-by-step https://foojay.io/today/building-project-panamas-jextract-tool-by-yourself/ But sadly when I use git bash in the downloaded jextract folder there occur 3 […]

Paulo Cabrita

Very nice article!

Just my 2 cents. You can’t execute the “sh ./gradlew…” having the JAVA 19 being the main JAVA of your machine. I will not work. I think you should have 2 JDK, one for executing the Gradle (with for instance the 18 version) and 19 JDK that we want to configure.

I saw this issue here: https://github.com/openjdk/jextract#building–testing

David

Hi Carl, great series of articles, thank you! I tried following your instructions on Apple Silicon, but get the following error when running gradle: “No signature of method: static java.nio.file.Path.of() is applicable for argument types: (String) values: [/Library/Developer/CommandLineTools/usr]
Possible solutions: is(java.lang.Object), join(java.lang.String), min(), max(), sort(), any()”

Do you have any idea what could be the cause (/Library/Developer/CommandLineTools/usr is the path to the default LLVM installation under Ventura; I also tried …/usr/lib and …/usr/lib/libclang.dylib).

If you have time. Thanks in advance.

Subscribe to foojay updates:

https://foojay.io/feed/
Copied to the clipboard