ElasticSearchDataProvider Module

Table of Contents

• Introduction to the ElasticSearchDataProvider Module
• ElasticSearch Data Provider Factory
• ElasticSearch Data Provider Examples
• Release Notes
  • ElasticSearchDataProvider v1.0

Qore ElasticSearchDataProvider module definition

Introduction to the ElasticSearchDataProvider Module

The ElasticSearchDataProvider module provides a data provider API for ElasticSearch instances and data. It provides introspection into the configuration and document attributes as well as APIs and a record-based data provider to use ElasticSearch indices as DB table-like objects as well.

This data provider provides ElasticSearch API access to:

• documents: document/{create, read, update, delete}
• indexes: index/{create, read, read-all, delete, search}
• pipelines: pipeline/{create, read, read-all, update, delete}

Furthermore it provides a record interface to ElasticSearch indexes supporting record creating, searching, updating, and deleting documents (where an ElasticSearch document is treated as a record) through the following path:

• indexes/index_name

These data provides represent a single index with a record-like interface; each document (record) will be returned with read-only _id and _score fields as well. Searches are performed with match; for flexible search criteria, use the API-driven search (ElasticSearchIndexSearchDataProvider)

In this way ElasticSearch can be used for easy data integration and searches as well as for the manipulation of document content and pipelines from the Data Provider API.

ElasticSearch Data Provider Factory

The name of the ElasticSearch data provider factory is elasticsearch.

ElasticSearch Data Provider Examples

These examples are with qdp, the command-line interface to the Data Provider API.

API Example: List All Pipelines

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/pipeline/read-all'

API Example: Create a Pipeline

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/pipeline/create' \
  pipeline=my-pipeline,description="my pipeline",processors='{html_strip={field=content}}'

API Example: Delete a Pipeline

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/pipeline/delete' pipeline=my-pipeline

API Example: List All Indices

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/index/read-all'

API Example: Create an Index

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/index/create' \
  index=my-index,mappings={properties={content={type=text}}}'

API Example: Delete an Index

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/index/delete' index=my-index

API Example: Search an Index

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/index/search' \
  index=my-index,'query={match={content=target-string}}'

API Example: Create a Document / Ingest Content

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/document/create' \
  index=my-index,pipeline=my-pipeline,content="<h2>my HTML content</h2>",refresh=true

API Example: Delete a Document

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/document/delete' \
  index=my-index,id=oE1jYokB_lAyoyNWRDP9

Record-Based Example: Search an Index

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/indexes/my-index' \
  content=target-string

Record-Based Example: Create a Document / Ingest Content

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/indexes/my-index' create \
  content="<h2>my HTML content</h2>" pipeline=my-pipeline,refresh=true

Record-Based Example: Delete a Document

qdp 'elasticsearch{url=https://elastic:password@my-server.local:9200}/indexes/my-index' delete \
  _id=oE1jYokB_lAyoyNWRDP9

Release Notes

ElasticSearchDataProvider v1.0

• initial release of the module