Qore jni Module 2.4.0
Loading...
Searching...
No Matches
Qore jni Module

Table of Contents

jni Module Introduction

The jni module allows for Java APIs to be used in Qore as if the Java APIs were Qore APIs and vice-versa, as Qore APIs can be dynamically imported into Java programs at runtime; additionally a custom Java compiler implementation is included that allows for building bytecode with the Qore class loader that supports dynamic byte code generation for Qore and Python language bindings; see Importing Qore and Python APIs Dynamically in Java for more information.

Native API imports and bindings as well as run-time data translations between Java and Qore and Java and Python are supported.

Java bytecode generation for dynamic API generation is provided by Byte Buddy, an excellent and flexible open-source API for Java bytecode generation.

Also included with the jni module:

This module is released under the MIT license (see COPYING.MIT in the source distribution for more information). The module is tagged as such in the module's header (meaning it can be loaded unconditionally regardless of how the Qore library was initialized).

To use the module in a Qore script, use the %requires directive as follows:

%requires jni

Helper Qore classes provided by this module:

Class Description
JavaArray a convenience class for using Java Arrays in Qore
QoreInvocationHandler a convenience class for executing Qore-language callbacks from Java

Helper Qore functions provided by this module:

Function Description
get_version() Returns the version of the JNI API
get_byte_code() Returns the dynamically generated Java byte code of the given Qore class
implement_interface() Creates a Java object that implements given interface using an invocation handler
invoke() Invokes a method with the given arguments
invoke_nonvirtual() Invokes a method with the given arguments in a non-virtual way; meaning that even if the object provided is a child class, the method given in the first argument is executed
load_class() Loads a Java class with given name and returns a java::lang::Class object
new_array() Creates a JavaArray object of the given type and size
set_save_object_callback() Sets the object lifecycle management callback; see Managing the Lifecycle of Qore objects from Java for more information

Examples

Java Dynamic Python Import Example:
// import the "xml" module and all symbols from xml.etree.ElementTree into Java
import pythonmod.xml.etree.ElementTree.*;
// extend the pythonmod.xml.etree.ElementTree.ElementTree class
public class QoreDynamicTest10 extends ElementTree {
// we need to declare that our constructor can throw an exception
public QoreDynamicTest10() throws Throwable {
}
// parse some XML and then return the value of the given element
public static Object test(String str, String name) throws Throwable {
Element elem = (Element)pythonmod.xml.etree.ElementTree.$Functions.fromstring(str);
Element e0 = elem.find(name);
Object rv = e0.getMemberValue("text");
return rv;
}
}
Java Dynamic Qore Import Example:
import java.util.Arrays;
import java.util.Iterator;
// import ::Qore::AbstractIterator as an abstract Java base class
import qore.Qore.AbstractIterator;
// this class wraps a Java Iterator as a Qore AbstractIterator
class QoreIteratorTest extends AbstractIterator {
private Iterator i;
QoreIteratorTest(Iterator i) throws Throwable {
this.i = i;
}
public boolean next() {
return i.hasNext();
}
public Object getValue() {
return i.next();
}
public boolean valid() {
return true;
}
}
Qore JMS Example:
#!/usr/bin/env qore
%new-style
%require-types
%strict-args
%enable-all-warnings
%requires jni
%requires QUnit
%requires Util
# import Java classes to our script
%module-cmd(jni) import javax.naming.InitialContext
%module-cmd(jni) import javax.jms.*
# add environment variable $GLASSFISH_JAR to the dynamic classpath (if set)
%module-cmd(jni) add-classpath $GLASSFISH_JAR
%exec-class Main
public class Main inherits QUnit::Test {
private {
Counter c(1);
auto data;
const TextMsg = "Hello, world!";
}
constructor() : Test("JMS test", "1.0", \ARGV) {
addTestCase("base test", \testJms());
# Return for compatibility with test harness that checks return value.
set_return_value(main());
}
testJms() {
# these properties are the default; included here to provide an example for connecting to a remote server
Properties props();
props.setProperty("java.naming.factory.initial", "com.sun.enterprise.naming.SerialInitContextFactory");
props.setProperty("org.omg.CORBA.ORBInitialHost", "localhost");
props.setProperty("org.omg.CORBA.ORBInitialPort", "3700");
InitialContext ctx(props);
Connection connection =
cast<ConnectionFactory>(ctx.lookup("jms/__defaultConnectionFactory")).createConnection();
Session session = connection.createSession(Session::CLIENT_ACKNOWLEDGE);
Destination queue = cast<Destination>(ctx.lookup("abc"));
MessageProducer producer = session.createProducer(queue);
MessageConsumer consumer = session.createConsumer(queue);
# in order to implement the MessageListener interface for the callback, we have to use implement_interface()
# as follows:
ClassLoader loader = connection.getClass().getClassLoader();
consumer.setMessageListener(cast<MessageListener>(implement_interface(loader,
new QoreInvocationHandler(\messageCallback()), Class::forName("javax.jms.MessageListener", True,
loader))));
connection.start();
TextMessage message = session.createTextMessage();
message.setText(TextMsg);
producer.send(message);
# wait for message to be received
c.waitForZero();
assertEq(TextMsg, data);
# unset the listener and exit
consumer.setMessageListener(NOTHING);
}
# JMS message callback for the MessageListener interface
messageCallback(Method method, *list args) {
TextMessage msg = args[0];
# ignore redeliveries
if (msg.getJMSRedelivered())
return;
data = msg.getText();
msg.acknowledge();
if (m_options.verbose)
printf("*** JMS message received: %y (id: %y)\n", msg.getText(), msg.getJMSMessageID());
c.dec();
}
}

JVM Initialization

The JVM is initialized when the module is loaded. It is possible to disable JIT when the module is loaded, however the module is loaded an initialized before module parse commands are processed, therefore to tell the module to disable JIT, the following environment variable must be set to 1:

  • QORE_JNI_DISABLE_JIT=1

Heap sizes can be controlled by the following environment variables:

  • QORE_JNI_MIN_HEAP_SIZE: will set the -Xms JVM initialization option for the minimum heap size
  • QORE_JNI_MAX_HEAP_SIZE: will set the -Xmx JVM initialization option for the maximum heap size

For example: 

# to initialize the JVM with a minimum and maximum heap size of 20MB
QORE_JNI_MIN_HEAP_SIZE=20m QORE_JNI_MAX_HEAP_SIZE=20m qore -l jni script.q

Signals Used By the JVM

The JVM requires signals to run properly, and, while Qore initializes the JVM with reduced signals (using the -Xrs argument to the JVM), certain signals are critical to the operation of the JVM and cannot be used by Qore programs.

The signals in the following table are reserved by the JVM.

Signals Reserved by the JVM

Signal Description
SIGTRAP Trap exception
SIGSEGV Segmentation violation
SIGBUS Bus error
SIGCHLD Child process terminated
SIGILL Illegal instruction
SIGFPE Floating-point exception

If any Qore code has a signal handler assigned to any of the above signals (for the signals that can be assigned), the jni module cannot be initialized and will fail to load.

Using Java APIs in Qore

Importing Java APIs into Qore

To import Java APIs into the Qore program; any of the following jni-module-specific parse directives can be used, each of which also causes the jni module to be loaded and initialized:

  • %module-cmd(jni) import java.namespace.path.*
    imports the given wilcard path or class into the Qore program
  • %module-cmd(jni) add-classpath filesystem/path:another/path
    adds the given paths to the Program-specific runtime dynamic classpath; supports environment variable substitution
  • %module-cmd(jni) add-relative-classpath ../relative/path
    adds the given paths as relative to the current program to the Program-specific runtime dynamic classpath
  • %module-cmd(jni) global-add-classpath filesystem/path:another/path
    adds the given paths to the global runtime dynamic classpath; supports environment variable substitution
  • %module-cmd(jni) global-add-relative-classpath ../relative/path
    adds the given paths as relative to the current program to the global runtime dynamic classpath

All classes in java.lang.* are imported implicitly. Referencing an imported class in Qore code causes a Qore class to be generated dynamically that presents the Java class. Instantiating a Qore class based on a Java class also instantiates an internal Java object that is attached to the Qore object. Calling methods on the object will cause the same Java methods to be called with any arguments provided.

See also
Java Class and Package Mapping in Qore for more information

Defining Java Classes at Parse Time in Qore

The jni module also supports defining classes at parse time with the following jni-module-specific parse directives:

  • %module-cmd(jni) define-class <name> <base 64 bytecode>
    defines the given class with <name> as the Java internal name for the class (ex: my/package/MyClassName)
  • %module-cmd(jni) define-pending-class <name> <base 64 bytecode>
    adds the given class and bytecode as a pending class to be defined when required by a define-class directive; this should be used for inner classes and has been implemented to allow circular dependencies in such classes to be resolved; <name> is the Java internal name for the inner class (ex: my/package/MyClassName$1)

Setting Java System Properties at Parse Time in Qore

The jni module also supports setting Java system properties at parse time with the following jni-module-specific parse directive:

  • %module-cmd(jni) set-property <propname> <value>
    sets the given property at parse time

Java Class and Package Mapping in Qore

Java Package to Qore Namespace Mapping

The jni module maps Java packages to Qore namespaces. Therefore in Java java.lang.String can be referred to as java::lang::String in Qore as in the following example:

java::lang::String str();

The jni module attempts to create Qore classes that match the source Java class as closely as possible to maximize the usefulness of Java APIs in Qore.

When creating Qore class methods from the corresponding Java source method, types are mapped according to Java to Qore Type Conversions. Because Qore has fewer types than Java, this can lead to method signature collissions in the generated Qore class.

In such cases, only the first method returned by the internal call to java.lang.Class.getDeclaredMethods() is mapped.

To call an unmapped method, use reflection to get a java.lang.Method object and make the call using java.lang.Method.invoke() as in the folllowing example:

StringBuilder stringBuilder("");
Method stringBuilderAppendLong = stringBuilderClass.getMethod("append", Long::TYPE);
stringBuilderAppendLong.invoke(stringBuilder, 20);
printf("%y\n", stringBuilder.toString());

Java Inner Classes in Qore

Java inner classes are mapped as Qore classes by appending "__" to the outer class's name and then appending the name of the inner class.

For example:

class Outer {
public Outer() {
}
public class Inner {
public Inner() {
}
}
}

The inner class would be created as "Outer__Inner" in Qore.

Subclassing Java Classes in Qore

When subclassing Java classes in Qore, the Qore class code is not executed by Java when dispatching method calls in Java itself. This means that overriding Java methods in Qore is only valid for code executed from Qore.

To override a Java method when called from Java, you must subclass the class in Java as well and import the child class into your Qore program.

One exception to this limitation is with interfaces; programmers can use the QoreInvocationHandler class to execute Qore-language callbacks based on Java interfaces; see the documentation for QoreInvocationHandler for more information.

Java Class Fields to Qore Class Mappings

Java fields are mapped to different Qore class members according to the Java type according to the following table.

Java Field to Qore Class Member Mappings

Java Qore Qore Example
static final Qore class constants
MyClass::myField
static (non-final) Qore static class members
obj.myField
all others normal Qore class members
obj.myField

For example:

System::out.println("testing...");
Note
Java field values are only stored in Java; they are not mirrored in Qore. Java field values are accessed with the class's memberGate() method; updating a Java object's field value in Qore will not cause the field value to be updated in Java. To update the field's value from Qore in Java, use java::lang::reflect::Field::set().

Java Access Mapping in Qore

Java access modifiers are mapped directly to Qore access modifiers as in the following table.

Java to Qore Access Mappings

Java Qore
public public
protected private (or private:hierarchy)
private private:internal

Java Classloader

There is a global classloader for all Java objects reachable without special classpath. For all other objects, each Qore Program container has its own custom classloader provided by the jni module that supports a dynamic classpath.

The Qore Program-specific dynamic classpath can be set with the import commands as documented in Importing Java APIs into Qore.

Explicit class loaders can also be used as in Java as in the following example:

softlist<URL> urls = (new URL("file:///my/dir/my-api.jar"));
URLClassLoader classLoader(urls);
Class myClass = classLoader.loadClass("MyClass");
Note
The class loader supplied by this module, org.qore.jni.QoreURLClassLoader, supports dynamic bytecode generation from Python and Qore imports.

Java Exceptions in Qore

If a org.qore.jni.QoreException exception is thrown, then it is mapped directly to a Qore exception.

Other Java exceptions are mapped to the Qore ExceptionInfo hash as follows:

  • err: always "JNI-ERROR"
  • desc: the full Java class name of the exception class; ex: "java.lang.RuntimeException"
  • arg: the exception object itself as a Qore object wrapping the Java object

Exception locations including call stack locations reflect the actual Java source location(s), and in such cases the lang attribute will be "Java".

Using the jni Module From Java

Java code can use the jni module and Java classes based on Qore code by calling org.qore.jni.QoreJavaApi.initQore() to initialize the Qore library and the jni module with an existing JVM session. This requires the platform-dependent qore library to be found in a directory set in the java.library.path or by setting the QORE_LIBRARY environment variable to the absolute path of the qore library.

If neither of these are set, then calling org.qore.jni.QoreJavaApi.initQore() will result in a java.lang.UnsatisfiedLinkError exception being raised.

All Java classes for Qore support are located in the qore-jni.jar file.

Example Java source:

import org.qore.lang.qunit.*;
class MyTest {
public static void main(String[] args) throws Throwable {
Test test = new Test("MyTest", "1.0");
test.addTestCase("test", () -> doTest(test));
test.main();
}
private static void doTest(Test test) throws Throwable {
test.assertTrue(true);
}
}

Example compilation command: 

    javac -cp qore-jni.jar MyTest.java

Example run command: 

    java -Djava.library.path=/usr/local/lib -cp qore-jni.jar:. MyTest

Importing Qore and Python APIs Dynamically in Java

Dynamic Qore and Python API imports are supported with qore-jni.jar at runtime; qore-jni-compiler.jar contains a compiler API that can build bytecode that supports providing dynamic imports to the Java compiler as well.

At runtime, only the qore-jni.jar file (and an accessible Qore shared library) are needed.

Dynamic Import Statements

The following special packages provide access to dynamically-generated classes:

  • python.[path...]: indicates that the given path should be imported from Python to Java (after being imported to Qore if necessary). Only use for Python symbols not provided by a Python module.
  • pythonmod.mod.[path...]: indicates that the given Python module should be loaded and imported into Qore and the given symbols should be imported into Java. Python symbols provided by modules should be referenced with import statements using pythonmod as the symbols imported into Java will use such package names as well.
  • qore: indicates that the given path should be mapped to Qore namespaces and/or classes; the Java package segments after qore. are then converted to the equivalent Qore namespace path; cannot be used for Qore symbols provided by modules
  • qoremod.mod.$[path...]: indicates that the given Qore module should be imported into the program container, and the given symbols should then be imported into Java. Qore symbols provided by modules must be referenced with import statements using qoremod as the symbols imported into Java will use such package names as well.

For example:

  • import pythonmod.json.JSONEncoder; would cause the json Python module to be loaded and imported into Qore and the JSONEncoder class to be imported into Java
  • import qore.Qore.Thread.ThreadPool; would cause the Qore ::Qore::Thread::ThreadPool class to be imported automatically into Java
  • import qoremod.RestClient.RestClient; would cause the RestClient Qore module to be loaded and the RestClient class to be imported into Java

Qore and Python classes imported as above can then be used as Java classes for instantiation or as base classes for Java subclasses.

Note
  • Qore acts as a "language bridge" between Python and Java; Python can also inherit Java classes using a similar mechanism; Java classes are imported into Qore and then the wrapper Qore classes are imported dynamically into Python
  • The pythonmod and qoremod magic packages are used to ensure that modules are imported both at compile and runtime and that the resulting dynamically-generated Java classes have consistent binary names
  • Symbols imported from Qore modules can be treated as non-module symbols by using the mark-module-injected module command; ex:
    %module-cmd(jni) mark-module-injected InjectedModule
    would mark all symbols from Qore module InjectedModule as non-module symbols
  • Qore and Python definitions imported into Java from the qoremod and pythonmod packages will be anchored under this package name even if definitions from other namespaces (from Qore imports) are imported
See also
Default Qore Object Lifecycle Management
Importing Functions Into Java

Qore and Python functions can be imported using the special $Functions class name; functions in a Qore namespace are imported automatically into a special class with this name as static methods in the corresponding dynamic Java package.

Importing Constants Into Java

Qore constants can be imported using the special $Constants class name; constants in a Qore namespace are imported automatically into a special class with this name as static final fields in the corresponding dynamic Java package.

Running Java Programs Using Dynamic Imports

You must use the org.qore.jni.QoreURLClassLoader class as the system classloader to run Java bytecode using dynamic imports.

For example: 

java -cp jars/my-jar.jar:jars/qore-jni.jar:. -Djava.system.class.loader=org.qore.jni.QoreURLClassLoader
        MyClass

In case the Qore shared library cannot be found automatically, you may need to set the QORE_LIBRARY environment variable to the library's location as in the following example: 

QORE_LIBRARY=/opt/qorus/lib/libqore.so java -cp jars/my-jar.jar:jars/qore-jni.jar:.
        -Djava.system.class.loader=org.qore.jni.QoreURLClassLoader MyClass

Using Dynamic API Imports While Compiling

The org.qore.jni.compiler.QoreJavaCompiler class must be used to compile bytecode using dynamic imports.

The standard javac program wil not work, because it is not capable of understanding or generating the bytecode for the dynamic imports in the special Java package.

Here is a Java example using the compiler:

import org.qore.jni.compiler.QoreJavaCompiler;
import org.qore.jni.compiler.CompilerOutput;
import java.nio.file.Files;
// compile and execute QoreDynamicTest.java
class MyCompiler {
public static void main(String[] args) {
QoreJavaCompiler compiler = new QoreJavaCompiler();
CompilerOutput newClassData = compiler.compile("org.qore.test.QoreDynamicTest",
Files.readString("./QoreDynamicTest.java", StandardCharsets.UTF_8));
// get byte code with: newClassData.file.openInputStream().readAllBytes()
Object obj = newClassData.cls.getDeclaredConstructor().newInstance();
// ...
}
}

And here is a Qore example:

#!/usr/bin/env qore
# -*- mode: qore; indent-tabs-mode: nil -*-
%new-style
%require-types
%strict-args
%enable-all-warnings
%requires jni
%module-cmd(jni) add-relative-classpath $MY_JARS/qore-jni.jar
%module-cmd(jni) add-relative-classpath $MY_JARS/qore-jni-compiler.jar
%module-cmd(jni) import org.qore.jni.compiler.QoreJavaCompiler
%module-cmd(jni) import org.qore.jni.compiler.CompilerOutput
# compile and execute QoreDynamicTest.java
QoreJavaCompiler compiler();
compiler::CompilerOutput newClassData = compiler.compile("org.qore.test.QoreDynamicTest",
File::readTextFile(get_script_dir() + "/QoreDynamicTest.java"));
# get byte code with: newClassData.file.openInputStream().readAllBytes();
Object obj = newClassData.cls.getDeclaredConstructor().newInstance();
# ...
org.qore.jni.compiler.CompilerOutput
Definition CompilerOutput.java:8
org.qore.jni.compiler.QoreJavaCompiler
Definition QoreJavaCompiler.java:74
Note
  • The qjavac helper script can be used to compile Java sources to class files; ex 
    qjavac -cp my-jar.jar MySource.java
  • The qjava2jar helper script can be used to compile Java sources using dynamic imports to a jar file; ex: 
    qjava2jar my-jar.jar source_path -cp some-api.jar:another-api.jar -nowarn

JNI Module Compatibility Options

This module supports the following compatibility option: "compat-types" which, when enabled, will disable the following type conversions:

  • Java byte[] to binary; instead this will converted to list<int>
  • Java java.util.Map to hash; instead this will be converted directly

This option can be set globally for all Program objects with set_module_option("jni", "compat-type", True) or locally for the current Program container with the set-compat-type module parse option like: module-cmd(jni) set-compat-type true or module-cmd(jni) set-compat-type false to override the global setting.

Type Conversions Between Qore and Java

The jni module uses reflection to automatically map Java classes to Qore classes. This class mapping and Qore class creation happens at parse time (when importing Java APIs into Qore) and also at runtime (if a new class is encountered that has not already been mapped).

There are two types of conversions from Qore to Java:

Qore to Java Defined Type Conversions

The following table describes type conversions for Java types when the jni module must convert a Qore value to the declared type:

Qore to Specific Java Type Conversions

Target Java Type Source Qore Type
boolean any; conversions are made like with boolean()
byte any; conversions are made like with int()
char any; conversions are made like with int()
short any; conversions are made like with int()
int any; conversions are made like with int()
long any; conversions are made like with int()
float any; conversions are made like with float()
double any; conversions are made like with float()

For other types, default conversions are used.

Qore to Java Default Type Conversions

The following table describes type conversions for Java types when the jni module must convert a Qore value to the declared type:

Qore to Java Default Type Conversions

Source Qore Type Target Java Type
bool boolean
int long
float double
string java.lang.String
date java.lang.Object (compile time) or java.time.ZonedDateTime or org.qore.jni.QoreRelativeTime (runtime)
number java.math.BigDecimal
binary byte[]
NOTHING and NULL void
list java Arrays of the list type; if no list type can be found, then the Array type is java.lang.Object
hash org.qore.jni.Hash (extends java.util.LinkedHashMap), however any type implementing java.util.Map that has a constructor that takes no arguments and a compatibule put() method will be constructed / converted automatically to the desired type
code (closure or call reference) org.qore.jni.QoreClosure
JavaArray java.lang.Array
object to the Java class if the Qore object is an instantiation of java.lang.Object, otherwise org.qore.jni.QoreObject
all jni objects direct conversion
Note
  • Qore 64-bit integers are automatically converted to 64-bit Java long values; to get a Java int value; use:
    new Integer(val)
  • see Java Arrays for more information about conversions between Qore lists and Java arrays

Java to Qore Type Conversions

Java to Qore Type Conversions

Source Java Type Target Qore Type
boolean bool
byte int
byte[] binary (see also JNI Module Compatibility Options)
char int
short int
int int
long int
float float
double double
java.lang.String string
java.sql.Date date (absolute date)
java.sql.Time date (absolute date based on 1970-01-01)
java.sql.Timestamp date (absolute date)
java.time.ZonedDateTime date (absolute date)
org.qore.jni.QoreRelativeTime date (relative date)
java.math.BigDecimal number
java.lang.AbstractArray and arrays list
java.util.Map hash (see also JNI Module Compatibility Options)
org.qore.jni.QoreClosure code
org.qore.jni.QoreClosureMarker code
all other objects direct conversion
Note
see Java Arrays for more information about conversions between Qore lists and Java arrays

Java Arrays

Arrays are mapped directly to and from Qore lists. When converting from a Qore list to a Java array, the list is scanned, and if it can be mapped to a single Java type, then an array of that type is created. Otherwise, an array of java.lang.Object is created and the Qore values are mapped directly to Java values.

Qore lists are passed by value and Java arrays are objects (which, like Qore objects, are passed with a copy of a reference to the object). To get similar functionality in Qore, you can use the JavaArray class supplied with the jni module, which is automatically converted to a Java array object like a list, but is also passed with a copy of a reference to the object in Qore.

Java Arrays as Variable Arguments

When a Java method declares variable arguments, such arguments must be generally given as a single array value (so a Qore list or JavaArray object). Consider the following example:

StringBuilder stringBuilder("");
lang::Class stringBuilderClass = lang::Class::forName("java.lang.StringBuilder");
Method stringBuilderAppendLong = stringBuilderClass.getMethod("append", (Long::TYPE,));
stringBuilderAppendLong.invoke(stringBuilder, (20,));
printf("%y\n", stringBuilder.toString());

In the above example, the java.lang.Class.getMethod() and the java.lang.Method.invoke() methods both take a variable number of arguments after the first argument. In the mapped Qore class, these must be given as a Qore list (as in the above example) or a JavaArray object.

If the last argument of a Java method has an array type, then the arguments in Qore can also be given in the flat argument list, and the final array argument will be created automatically for the call.

Because of this, the example above also works as follows:

StringBuilder stringBuilder("");
lang::Class stringBuilderClass = lang::Class::forName("java.lang.StringBuilder");
Method stringBuilderAppendLong = stringBuilderClass.getMethod("append", Long::TYPE);
stringBuilderAppendLong.invoke(stringBuilder, 20);
printf("%y\n", stringBuilder.toString());

Managing the Lifecycle of Qore objects from Java

Qore's deterministic garbage collection approach and reliance on destructors means that Qore objects created by Java must have their lifecycles managed externally.

Java objects wrapping Qore objects (such as org.qore.jni.QoreObject and org.qore.jni.QoreObjectWrapper) hold only weak references to Qore objects.

Java methods in the jni module that explicitly save strong references to Qore objects are as follows:

The strong reference to any Qore object returned by the preceding methods is managed in one of two ways described in the following sections.

Default Qore Object Lifecycle Management

By default, Qore objects are saved in thread-local data, so the lifecycle of the object is automatically limited to the existence of the thread.

The thread-local hash key name used to save the list of objects created is determined by the value of the "_jni_save" thread-local key, if set. If no such key is set, then "_jni_save" is used instead as the literal key for saving the list of objects.

Note
To register a Java thread in Qore and therefore to ensure that Qore object data is persistent in a native Java thread, call org.qore.jni.QoreJavaApi.registerJavaThread() and then org.qore.jni.QoreJavaApi.deregisterJavaThread() before the Java thread terminates, when you are done working with Qore data, otherwise Qore objects will be immediately deleted when the implicit Qore thread registration expires after each Qore API call.

Explicit Qore Object Lifecycle Management

The lifecycle of Qore objects can be managed explicitly by using the set_save_object_callback() function to set a callback that is called every time a Qore object is created using the Java API.

This callback must take a single object argument as in the following example:

hash<string, object> object_cache;
code callback = sub (object obj) {
# save object in object cache, so it doesn't go out of scope
object_cache{obj.uniqueHash()} = obj;
}
set_save_object_callback(callback);

Java Thread Stack size

The Java thread stack size is set when the JVM is initialized in the jni module's initialization code based on the Qore thread stack size.

Note that the JVM establishes its own stack guard implementation and enforces stack sizes, even for Qore code and native threads, as a result, the Qore thread stack size limitation is also applied to the primary process thread, which otherwise (at least on Linux, for example), normally has a larger limit (normally 8MB on Linux).

Therefore if the Qore thread stack size is set to 512KB (for example), the Java thread stack size limit as well as the primary thread's stack size limit is set to 512KB when the jni module is initialized.

The Qore library's default thread stack size is 8MB. This is much larger than the standard 1MB stack size for the JVM, however, since thread stacks are generally allocated from overcommitted memory, meaning that they start small and only grow on demand, a larger thread stack size does not imply a larger memory footprint for programs that do not require large stacks.

jdbc DBI Driver for Qore

This module implements the "jdbc" driver for Qore to allow Qore (as well as other languages such as Python that use Qore as a "language bridge") to access data in databases with jdbc drivers.

To use the jdbc DBI driver, use the driver name "jdbc".

Qore datasource strings do not follow the same format as standard Java JDBC URL strings, so some care must be taken when creating the Qore datasource strings for the jdbc driver.

The format for Qore datasource strings with the jdbc driver is as follows: jdbc:[user]/[pass]@db[{option=val[,...]}]
where all elements except @db are optional.

The host, port, and charset values in Qore datasource strings are ignored; the host and port must be given in the db value, and, if the jdbc driver's URL format requires the use of @ or : characters, then a dummy value must be provided for the db value, and the url option must be used instead.

Some example Qore datasource strings for the jdbc driver:

  • jdbc:user/pass@postgresql:dbname{classpath=/usr/share/java/postgresql.jar}
  • jdbc:user/pass@firebird:?serverName=hq&databaseName=/var/lib/firebird/data/mydb.fdb&encoding=WIN1252{classpath=/usr/share/java/jna-5.12.1.jar:/usr/share/java/jaybird-5.0.0.java11.jar}
  • jdbc:user/pass@ignored{url=oracle:thin:@hostname:1521:dbserver,classpath=/usr/lib/oracle/21/client64/lib/ojdbc8.jar}
Note
the initial jdbc: in the db value is optional; if missing, it is added automatically by the Qore jdbc driver when the internal URL string is passed to the internal Java JDBC call.

jdbc DBI Driver Options

The jdbc driver supports the following DBI options:

  • "classpath": sets the classpath with jar or class files providing the jdbc driver
  • "numeric-numbers": return received SQL_NUMERIC and SQL_DECIMAL values as arbitrary-precision numbers (Qore number values)
  • "optimal-numbers": return received SQL_NUMERIC and SQL_DECIMAL values as integers if possible, if not return them as an arbitrary-precision numbers; this is the default
  • "string-numbers": return received SQL_NUMERIC and SQL_DECIMAL values as strings (for backwards- compatibility)
  • "url": sets the URL for the jdbc driver in case the database value in the Qore datasource connection string cannot accommodate the jdbc URL because of special characters
Note
By default optimal-numbers is set on the driver for maximum compatibility with other Qore DBI drivers

jdbc classpath Option

The classpath option allows for the classpath to be set before the JVM loads the Java jdbc driver. The value of this option must be a string and is subject to environment variable substitution before being used.

If this option is used, then the java.util.ServiceLoader for "java.sql.Driver" objects is run and iterated after updating the classpath, to ensure that any new drivers are loaded and available to drivers loaded after the initial driver is loaded, otherwise the java.sql.DriverManager will not see the new drivers as they were not present in the classpath during on-demand static initialization).

jdbc url Option

The url option allows for jdbc URLs to be used that contain special characters like @ and : that would cause the datasource to fail to parse when used as a database name.

url Option Example
    Datasource ds("jdbc:user/pass@ignored{url=oracle:thin:@hostname:1521:dbserver,classpath=/usr/lib/oracle/21/client64/lib/ojdbc8.jar}");
Note
the initial jdbc: in the url option is optional; if missing, then it is added automatically by the Qore jdbc driver when the internal URL string is passed to the internal Java JDBC call.

jni Module Release Notes

jni Module Version 2.4.0

  • added the ExcepDataProvider module to allow for reading record-based data from Excel spreadheets
  • MqttDataProvider module
    • added support for the data provider action catalog API (issue 4808)
  • fixed handling conversions from Qore and Python to Java for Float values (issue 4890)
  • fix for importing Qore methods with varargs into Java (issue 4833)

jni Module Version 2.3.1

  • fixed a bugs handling the signal mask that could cause hard-to-debug problems such as deadlocks in JIT in the JVM on macOS aarch64 (issue 4793)
  • fixed signatures for Java methods with varargs params (issue 4787)
  • fixed a bug where Qore's thread stack size settings were not applied to Java threads (issue 4769)

jni Module Version 2.3.0

jni Module Version 2.2.1

  • fixed a bug handling class loaders in JDBC connection acquisition when called from Qore code imported in native Java code; in this case dynamic dispatch is used in the internal call to DriverManager.getConnection() to ensure that the call succeeds (issue 4740)

jni Module Version 2.2.0

  • implemented support for specifying the minimum an maximum heap sizes for the JVM on initialization from environment variables (issue 4734)
  • fixed bugs handling dynamic Java class generation from Qore classes with abstract methods (issue 4737)

jni Module Version 2.1.1

  • fixed reading date/time values from MS SQL Server databases (issue 4705)
  • fixed exception handling in destroying JNI program data (issue 4704)

jni Module Version 2.1.0

  • added support for the jdbc DBI driver (issue 4684)
  • fixed a bug where qjavac did not detect the binary path of compiled objects correctly in all cases (issue 4683)
  • fixed a bug in qjavac where error information was not output when compiler diagnostic information was missing (issue 4682)

jni Module Version 2.0.11

  • updated to work with the latest qpp from Qore 1.12.4+

jni Module Version 2.0.10

jni Module Version 2.0.9

jni Module Version 2.0.8

  • fixed a bug importing Qore abstract methods with default or optional arguments (issue 4570)

jni Module Version 2.0.7

  • fixed a bug where missing c++ exception handling could cause a crash (issue 4500)

jni Module Version 2.0.6

  • fixed a bug where qjavac wrote output files in the current working directory instead of in the directory of the input file (issue 4376)
  • fixed a bug where Qore and Python functions could not be imported into the special $Functions Java class if the enclosing namespace had less than two classes in it (issue 4375)
  • fixed a bug where org.qore.jni.QoreURLClassLoader could not be used as the system class loader (issue 4374)
  • fixed a bug where Qore and Python dynamic module imports were not created with consistent Java binary names; also to facilitate handling injected symbols, the mark-module-injected module command was implemented (issue 4373)
  • fixed a bug where Java method args in array format were not always converted from Qore values correctly (issue 4372)
  • fixed a bug where the Qore Program and Java ClassLoader context was not always set correctly when instantiating Qore classes from Java when there were dependencies on dynamic classes, leading to class resolution errors (issue 4371)

jni Module Version 2.0.5

  • fixed a memory error in exception handling in Qore to Java tyype conversions that could cause a core dump (issue 4349)

jni Module Version 2.0.4

  • fixed a bug handling Qore callbacks in native Java threads (issue 4340)
  • fixed a bug handling Java interfaces as parents of a Java class (issue 4337)

jni Module Version 2.0.3

  • implemented the magic '$' target when importing Qore or Python modules with import statements with the qoremod or pythonmod special packages (issue 4304)

jni Module Version 2.0.2

  • fixed a bug where vararg arguments were not handled correctly when dynamically-generated Java called a Qore or Python static method (issue 4302)
  • fixed a bug where a runtime error was generated when used with Java 17 or above (issue 4300)

jni Module Version 2.0.1

  • fixed a bug where parameters of Qore type null or nothing would cause runtime errors when importing the method into Java (issue 4287)
  • fixed static method calls of imported Qore methods in Java; the calling context is now set correctly (issue 4286)
  • fixed importing abstract Qore methods into Java; Java methods are no longer marked as taking trailing vararg arguments (issue 4282)
  • fixed importing Qore classes with multiple inheritance into Java; methods from other non-primary base classes are now also imported into the dynamic Java class (issue 4091)

jni Module Version 2.0

  • Qore int values are now converted to Java long in order to maintain precision
  • the old hand-developed Java wrapper classes are deprecated as of this release in favor of dynamic imports
  • implemented support for dynamically importing Qore code into Java (issue 4091)

jni Module Version 1.2

  • fixed crashes handling deleted Qore program contexts (issue 4262)
  • fixed crashes handling Java lambdas wrapping Qore closures and call references (issue 4261)
  • fixed importing AWT classes in headless mode (issue 4231)
  • fixed method calls for methods that return base types where a caller context is required (for Reflection access control, for example) (issue 4093)
  • fixed a memory leak (or an assertion crash in debug mode) for a duplicate member error when importing Java classes in Qore (issue 4029)
  • added support for converting Qore closures / call references to a Java object (issue 3995)

jni Module Version 1.1.3

jni Module Version 1.1.2

  • use java.util.List<?> instead of java.util.ArrayList<?> as the base class for list conversions (issue 3873)
  • added the org.qore.jni.Hash class to use when passing Qore hashes to Java to make it easier to work with Qore hash data (issue 3868)
  • fixed a crash in Qore Program setup when initiated from a Qore thread with no Java context thread (issue 3862)
  • fixed a crash in method invocation due to incorrect detection of the Java thread context (issue 3794)
  • fixed a bug in SmtpClient.sendMessage()

jni Module Version 1.1.1

jni Module Version 1.1

jni Module Version 1.0.1

  • fixed setting the class loader context when calling Qore code from Java threads (issue 3585)

jni Module Version 1.0

  • initial public release