dkutils.streamsets_api.datacollector_client module

class dkutils.streamsets_api.datacollector_client.DataCollectorClient(host, port, username, password)

Bases: object

Client object for invoking StreamSets Data Collector REST API

Parameters

• host (str) – The name of the host to use in making REST API calls.

• port (int) – The port to use in making REST API calls.

• username (str) – Username to authenticate when making REST API calls.

• password (str) – Password to authenticate when making REST API calls.

__init__(host, port, username, password)

Client object for invoking StreamSets Data Collector REST API

Parameters

• host (str) – The name of the host to use in making REST API calls.

• port (int) – The port to use in making REST API calls.

• username (str) – Username to authenticate when making REST API calls.

• password (str) – Password to authenticate when making REST API calls.

get_pipeline_full_status(pipeline_id)

Get the JSON containing the status of the pipeline with the given pipeline_id

Parameters

pipeline_id (str) – The pipeline id of the pipeline for which status information is being requested

Raises

• HTTPError – If the request fails.

• ValueError – If the pipeline_id is None

Return type

requests.Response.json()

get_pipeline_status(pipeline_id)

Get an instance of Pipeline.Status that represents the status of the pipeline with the given pipeline_id

Parameters

pipeline_id (str) – The pipeline id of the pipeline for which status information is being requested

Raises

• HTTPError – If the request fails.

• ValueError – If the pipeline_id is None

Returns

PipelineStatus object

Return type

PipelineStatus

reset_pipeline(pipeline_id)

Resets the origin offset of the pipeline with the given pipeline_id

Parameters

pipeline_id (str) – The pipeline id of the pipeline for which status information is being requested

Raises

• HTTPError – If the request fails.

• ValueError – If the pipeline_id is None

start_pipeline(pipeline_id, **kwargs)

Start the pipeline with the given pipeline_id

Parameters

• pipeline_id (str) – The pipeline id of the pipeline to start

• **kwargs (dict) – Arbitrary keyword arguments to construct runtime parameters

Raises

• HTTPError – If the request fails.

• ValueError – If the pipeline_id is None

Return type

requests.Response.json()

stop_pipeline(pipeline_id, **kwargs)

Stop the pipeline with the given pipeline_id

Parameters

• pipeline_id (str) – The pipeline id of the pipeline to stop

• **kwargs (dict) – Arbitrary keyword arguments to construct runtime parameters

Raises

• HTTPError – If the request fails.

• ValueError – If the pipeline_id is None

Return type

requests.Response.json()

class dkutils.streamsets_api.datacollector_client.PipelineStatus(value)

Bases: enum.Enum

An enumeration.

CONNECTING = 'CONNECTING'

CONNECT_ERROR = 'CONNECT_ERROR'

DISCONNECTED = 'DISCONNECTED'

EDITED = 'EDITED'

FINISHED = 'FINISHED'

FINISHING = 'FINISHING'

RETRY = 'RETRY'

RUNNING = 'RUNNING'

RUNNING_ERROR = 'RUNNING_ERROR'

RUN_ERROR = 'RUN_ERROR'

STARTING = 'STARTING'

STARTING_ERROR = 'STARTING_ERROR'

START_ERROR = 'START_ERROR'

STOPPED = 'STOPPED'

STOPPING = 'STOPPING'

STOPPING_ERROR = 'STOPPING_ERROR'

STOP_ERROR = 'STOP_ERROR'