Qore DataProvider Module Reference 2.7.5
Loading...
Searching...
No Matches
DataProvider::AbstractDataProcessor Class Referenceabstract

Defines an abstract class for accepting data and outputting optionally transformed or filtered data. More...

#include <AbstractDataProcessor.qc.dox.h>

Public Member Methods

 flush (code enqueue)
 This method is called for each pipeline element to permit flushing of any pending data at the end of processing. More...
 
AbstractDataProviderType getExpectedType ()
 Returns the expected type of data to be submitted, if available. More...
 
AbstractDataProviderType getReturnType ()
 Returns the type of data that will be returned, if available. More...
 
 setThreadLocalData (*hash< auto > thread_local_data)
 Sets thread-local data to set before running the processor.
 
 submit (code enqueue, auto _data)
 Submits the data for processing. More...
 
bool supportsBulkApi ()
 Returns True if the data processor supports bulk operation. More...
 

Private Member Methods

 flushImpl (code enqueue)
 This method is called for each pipeline element to permit flushing of any pending data at the end of processing. More...
 
*AbstractDataProviderType getExpectedTypeImpl ()
 Returns the expected type of data to be submitted, if available. More...
 
*AbstractDataProviderType getReturnTypeImpl ()
 Returns the type of data that will be returned, if available. More...
 
abstract submitImpl (code enqueue, auto _data)
 Submits the data for processing. More...
 
abstract bool supportsBulkApiImpl ()
 Returns True if the data processor supports bulk operation. More...
 

Private Attributes

*hash< auto > thread_local_data
 Thread-local data to set before running the processor.
 
*list< string > thread_local_data_keys
 Thread-local data keys from thread_local_data.
 

Detailed Description

Defines an abstract class for accepting data and outputting optionally transformed or filtered data.

Member Function Documentation

◆ flush()

DataProvider::AbstractDataProcessor::flush ( code  enqueue)

This method is called for each pipeline element to permit flushing of any pending data at the end of processing.

Parameters
enqueuea closure taking a single argument that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated, then call it once for each generated record; prototype:
code enqueue = sub (auto qdata) {}
Note
  • Calls flushImpl() on the data to do the actual processing
  • Accept and return type information is not enforced in this method; it must be enforced in flushImpl()

◆ flushImpl()

DataProvider::AbstractDataProcessor::flushImpl ( code  enqueue)
private

This method is called for each pipeline element to permit flushing of any pending data at the end of processing.

Parameters
enqueuea closure taking a single argument that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated, then call it once for each generated record; prototype:
code enqueue = sub (auto qdata) {}
Note
  • The default implementation in the base class does nothing; to submit pending data for processing by subsequent elements in the pipeline when pipeline processing is finished, call enqueue with the pending data
  • No data should be cached or pending after this call; if the object requires closing or purging of internal state to make it reusable, it should be performed in this method

◆ getExpectedType()

AbstractDataProviderType DataProvider::AbstractDataProcessor::getExpectedType ( )

Returns the expected type of data to be submitted, if available.

Returns
the expected type of data to be submitted, if available
Note
Calls getExpectedTypeImpl() to provide the return value

◆ getExpectedTypeImpl()

*AbstractDataProviderType DataProvider::AbstractDataProcessor::getExpectedTypeImpl ( )
private

Returns the expected type of data to be submitted, if available.

This base class method returns NOTHING; reimplement in subclasses to provide a type

◆ getReturnType()

AbstractDataProviderType DataProvider::AbstractDataProcessor::getReturnType ( )

Returns the type of data that will be returned, if available.

Returns
the type of data that will be returned, if available
Note
Calls getReturnTypeImpl() to provide the return value

◆ getReturnTypeImpl()

*AbstractDataProviderType DataProvider::AbstractDataProcessor::getReturnTypeImpl ( )
private

Returns the type of data that will be returned, if available.

This base class method returns NOTHING; reimplement in subclasses to provide a type

◆ submit()

DataProvider::AbstractDataProcessor::submit ( code  enqueue,
auto  _data 
)

Submits the data for processing.

Parameters
enqueuea closure taking a single argument that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated from the input data, then call it once for each generated record; prototype:
code enqueue = sub (auto qdata) {}
_datathe data to process

Sets thread-local data before running and clears on exit if any is set

Note
  • Calls submitImpl() on the data to do the actual processing
  • Accept and return type information is not enforced in this method; it must be enforced in submitImpl()

◆ submitImpl()

abstract DataProvider::AbstractDataProcessor::submitImpl ( code  enqueue,
auto  _data 
)
privatepure virtual

Submits the data for processing.

Parameters
enqueuea closure taking a single argument that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated from the input data, then call it once for each generated record; prototype:
code enqueue = sub (auto qdata) {}
_datathe data to process

◆ supportsBulkApi()

bool DataProvider::AbstractDataProcessor::supportsBulkApi ( )

Returns True if the data processor supports bulk operation.

Returns
True if the data processor supports bulk operation
Note
Calls supportsBulkApiImpl() to return the answer

◆ supportsBulkApiImpl()

abstract bool DataProvider::AbstractDataProcessor::supportsBulkApiImpl ( )
privatepure virtual

Returns True if the data processor supports bulk operation.

Returns
True if the data processor supports bulk operation
See also
Data Provider Pipeline Bulk Processing