Qore Data

Table of Contents

• Qore Data
  • NaN-Boxing Encoding
  • QoreValue Initialization
  • Simple Value Types: QoreStringNode, DateTimeNode, BinaryNode
  • Special Value Types: NOTHING and NULL
  • Container Value Types: QoreHashNode, QoreListNode, QoreObject
  • Object Value Type: QoreObject
  • Handling Reference Arguments: ReferenceNode and QoreTypeSafeReferenceHelper

Qore Data

Data in Qore is handled with QoreValue, an 8-byte NaN-boxed value type that uses IEEE 754 double-precision NaN bit patterns to encode multiple value types in a single 64-bit word.

NaN-Boxing Encoding

QoreValue uses NaN-boxing to efficiently store the following types inline (no heap allocation):

• bool (true/false stored as special values)
• int64 (48-bit signed integers stored inline; larger integers use QoreBigIntNode)
• double (all IEEE 754 doubles including NaN, Inf, -0.0)
• Short strings (up to 6 UTF-8 bytes stored inline)
• Special values (NOTHING, NULL)
• Pointers to AbstractQoreNode (for heap-allocated complex types)

This encoding provides significant performance benefits:

• 50% memory reduction (8 bytes vs 16 bytes)
• Type checking is ~2x faster (simple bit comparisons)
• Short string comparison is ~18x faster (single uint64 compare)
• Function calls are 10-20% faster (single register vs two)

Note

The integer range for inline storage is +/-140,737,488,355,328 (+/-2^47). Integers outside this range are transparently stored as QoreBigIntNode pointers, which is invisible to user code.

All complex values in Qore are descended from AbstractQoreNode.

The following are the basic data types in Qore, implemented as C++ classes descended from AbstractQoreNode, and their type codes, accessed via AbstractQoreNode::getType():

• QoreStringNode (NT_STRING) for the string data type
• DateTimeNode (NT_DATE) for the date/time data type
• BinaryNode (NT_BINARY) for the opaque binary data type
• QoreNothingNode (NT_NOTHING) for a type signifying no value (also equivalent to nullptr!)
• QoreNullNode (NT_NULL) for the SQL NULL data type
• QoreHashNode (NT_HASH) for an associative array or hash data type
• QoreListNode (NT_LIST) for a list or array data type
• QoreObject (NT_OBJECT) for the data type representing the instantiation of a Qore class (QoreClass)

Additionally, the following classes are exposed in the library:

• ReferenceNode (NT_REFERENCE)
• AbstractFunctionReferenceNode (NT_FUNCREF)

See also

node_types.h for a complete list of all type codes.

The fastest way to directly access data of a specific type is to use QoreValue::getType() and then the QoreValue::get() template as follows:

if (val.getType() == NT_DATE) {
   DateTimeNode* dt = val.get<DateTimeNode*>();
   // .. do something with dt
}

For inline types, use the type-specific fast extraction methods:

// For integers (fast path for inline 48-bit integers)
if (val.getType() == NT_INT) {
   int64 i = val.getAsBigInt();  // works for both inline and QoreBigIntNode
}
 
// For floats
if (val.isFloat()) {
   double d = val.getAsFloat();
}
 
// For booleans
if (val.isBool()) {
   bool b = val.getAsBool();
}
 
// For short strings (inline strings up to 6 bytes)
if (val.isShortString()) {
   char buf[7];
   val.getShortString(buf);  // extracts the inline string
}

QoreValue Initialization

Warning

With NaN-boxing, uninitialized QoreValue variables contain garbage bits that may be misinterpreted as float values. Always use value-initialization for local QoreValue variables:

QoreValue val{};           // Correct: value-initialized to NOTHING
QoreValue val = QoreValue(); // Also correct
QoreValue val;             // DANGEROUS: uninitialized bits (OK only in unions or with placement new)

For class members, always use brace initialization in the member declaration:

class MyClass {
    QoreValue member{};    // Correct: will be NOTHING when default-constructed
    QoreValue bad_member;  // DANGEROUS: garbage bits if not initialized in constructor
};

Simple Value Types: QoreStringNode, DateTimeNode, BinaryNode

The QoreStringNode (NT_STRING), DateTimeNode (NT_DATE), and BinaryNode (NT_BINARY) classes are reference counted and can only be dynamically allocated.

They are all descendents of SimpleQoreNode, meaning that dereferencing their objects cannot cause a Qore-language exception to be thrown (hence all support the SimpleQoreNode::deref() function), therefore the SimpleRefHolder class can be used to manage temporary references to objects of these types.

For example:

{
   // here getReferencedQoreStringNode() returns a QoreStringNode value with an incremented reference count
   SimpleRefHolder<QoreStringNode> qstr(getReferencedQoreStringNode());
   printf("the result is: %s\n", qstr->c_str());
   // when qstr goes out of scope, the reference count is decremented by calling SimpleQoreNode::deref()
}

Special Value Types: NOTHING and NULL

With NaN-boxing, NOTHING and NULL are stored as special inline bit patterns in QoreValue, not as pointers to singleton objects. This makes testing for these values very efficient.

QoreValue val{};  // NOTHING
if (val.isNothing()) {
    // val is NOTHING
}
if (val.isNull()) {
    // val is NULL
}
if (val.isNullOrNothing()) {
    // val is either NULL or NOTHING
}

For backward compatibility with code using AbstractQoreNode pointers, the QoreNullNode and QoreNothingNode classes still exist, but when stored in a QoreValue, they are converted to the inline representation.

Note

When working with QoreValue directly, prefer using val.isNothing() over is_nothing(val.getInternalNode()) as the former is much faster (single bit comparison vs function call and pointer dereference).

The legacy global objects Null and Nothing still exist for compatibility with older code that works with AbstractQoreNode pointers directly.

Container Value Types: QoreHashNode, QoreListNode, QoreObject

The QoreHashNode (NT_HASH), QoreListNode (NT_LIST), and QoreObject (NT_OBJECT) classes define container types in Qore. QoreObject objects in particular could throw an exception when dereferenced (if the object goes out of scope and its destructor is run, the destructor could throw an exception). Because container types can hold any type, when they are deferenced it could cause a QoreObject to go out of scope, and therefore the possibility that a Qore-language exception could be thrown must be taken into consideration. Therefore, to dereference these objects a pointer to an ExceptionSink object must be passed to AbstractQoreNode::deref().

The ReferenceHolder class can be used to manage temporary reference counts as follows:

{
   // here a QoreHashNode value is returned with an incremented reference count
   // note that xsink must be "ExceptionSink *"
   ReferenceHolder<QoreHashNode> qhash(getQoreHashNode(), xsink);
   printf("there are %ld elements in the hash\n", qhash->size());
   // when qhash goes out of scope, the reference count is decremented
}

Object Value Type: QoreObject

QoreObject objects have node type NT_OBJECT as returned by AbstractQoreNode::getType().

QoreObject is special in that the implementation for objects in Qore mirrors that of Java, in that objects are passed by reference when used as function or method arguments, unlike other types. However, like Java, the reference to the object is passed, and not the value, so, while an object passed as an argument to a function can be modified by that function (modifications are made to the original object), in order to write a swap method, for example, you would need to pass the variables by reference (or the lvalue expression by reference, as the case may be) to the swap function. Unlike Java, Qore does support passing arguments by reference.

Handling Reference Arguments: ReferenceNode and QoreTypeSafeReferenceHelper

Handling lvalue references is more complicated, as access to global variables and object member references must be made under the appropriate thread locks. However the QoreTypeSafeReferenceHelper class makes access to lvalue references much easier and takes care of all the locking, access to the lvalue expression, as well as type enforcement. With the QoreTypeSafeReferenceHelper class you can get the type of the lvalue expression's value, get a pointer to a node with a reference count of 1 for in-place modification, or assign a new value to the lvalue.

Here is an example of the use of QoreTypeSafeReferenceHelper:

// *string chomp(reference<string> str) {}
static QoreValue f_chomp(const QoreListNode* args, RuntimeConfig& rc, ExceptionSink* xsink) {
    const ReferenceNode* str = HARD_QORE_VALUE_REF(args, 0);
    QoreTypeSafeReferenceHelper ref(str, xsink);
    if (*xsink) {
        return QoreValue();
    }
    assert(ref && ref.getType() == NT_STRING);
    QoreStringNode* rv = reinterpret_cast<QoreStringNode*>(ref.getUnique(xsink));
    if (*xsink) {
        return QoreValue();
    }
 
    rv->chomp();
    return rv->refSelf();
}