Skip to content

jobs

Jobs management controller that supports starting new jobs, such as running selected data quality checks

cancel_job

Cancels a running job
Source code

DELETE 

http://localhost:8888/api/jobs/jobs/{jobId}

Parameters of this method are described below

 Property name   Description                       Data type   Required 
job_id Job id string

Usage examples 

curl -X DELETE http://localhost:8888/api/jobs/jobs/123123124324324^
    -H "Accept: application/json"

collect_statistics_on_data_groups

Starts a new background job that will run selected data statistics collectors on tables, calculating separate metric for each data grouping
Source code

POST 

http://localhost:8888/api/jobs/collectstatistics/withgrouping

Return value

 Property name   Description                       Data type 
collect_statistics_queue_job_result CollectStatisticsQueueJobResult

Parameters of this method are described below

 Property name   Description                       Data type   Required 
job_business_key Optional job business key that is a user assigned unique job id, used to check the job status by looking up the job by a user assigned identifier, instead of the DQOps assigned job identifier. string
wait Wait until the statistic collection job finishes to run, the default value is false (queue a background job and return the job id) boolean
wait_timeout The wait timeout in seconds, when the wait timeout elapses and the job is still running, only the job id is returned without the results. The default timeout is 120 seconds, but could be reconfigured (see the 'dqo' cli command documentation). long

Request body

 Description                       Data type   Required 
Data statistics collectors filter StatisticsCollectorSearchFilters

Usage examples 

curl -X POST http://localhost:8888/api/jobs/collectstatistics/withgrouping^
    -H "Accept: application/json"^
    -H "Content-Type: application/json"^
    -d^
    "{\"connection\":\"sample_connection\",\"fullTableName\":\"sample_schema.sample_table\",\"enabled\":true,\"columnNames\":[\"sample_column\"],\"collectorCategory\":\"sample_category\"}"

collect_statistics_on_table

Starts a new background job that will run selected data statistics collectors on a whole table
Source code

POST 

http://localhost:8888/api/jobs/collectstatistics/table

Return value

 Property name   Description                       Data type 
collect_statistics_queue_job_result CollectStatisticsQueueJobResult

Parameters of this method are described below

 Property name   Description                       Data type   Required 
job_business_key Optional job business key that is a user assigned unique job id, used to check the job status by looking up the job by a user assigned identifier, instead of the DQOps assigned job identifier. string
wait Wait until the statistic collection job finishes to run, the default value is false (queue a background job and return the job id) boolean
wait_timeout The wait timeout in seconds, when the wait timeout elapses and the job is still running, only the job id is returned without the results. The default timeout is 120 seconds, but could be reconfigured (see the 'dqo' cli command documentation). long

Request body

 Description                       Data type   Required 
Data statistics collectors filter StatisticsCollectorSearchFilters

Usage examples 

curl -X POST http://localhost:8888/api/jobs/collectstatistics/table^
    -H "Accept: application/json"^
    -H "Content-Type: application/json"^
    -d^
    "{\"connection\":\"sample_connection\",\"fullTableName\":\"sample_schema.sample_table\",\"enabled\":true,\"columnNames\":[\"sample_column\"],\"collectorCategory\":\"sample_category\"}"

delete_stored_data

Starts a new background job that will delete stored data about check results, sensor readouts etc.
Source code

POST 

http://localhost:8888/api/jobs/deletestoreddata

Return value

 Property name   Description                       Data type 
delete_stored_data_queue_job_result DeleteStoredDataQueueJobResult

Parameters of this method are described below

 Property name   Description                       Data type   Required 
job_business_key Optional job business key that is a user assigned unique job id, used to check the job status by looking up the job by a user assigned identifier, instead of the DQOps assigned job identifier. string
wait Wait until the import tables job finishes to run, the default value is false (queue a background job and return the job id) boolean
wait_timeout The wait timeout in seconds, when the wait timeout elapses and the delete stored data job is still running, only the job id is returned without the results. The default timeout is 120 seconds, but could be reconfigured (see the 'dqo' cli command documentation). long

Request body

 Description                       Data type   Required 
Delete stored data job parameters DeleteStoredDataQueueJobParameters

Usage examples 

curl -X POST http://localhost:8888/api/jobs/deletestoreddata^
    -H "Accept: application/json"^
    -H "Content-Type: application/json"^
    -d^
    "{\"connection\":\"sample_connection\",\"fullTableName\":\"sample_schema.sample_table\",\"deleteErrors\":true,\"deleteStatistics\":true,\"deleteCheckResults\":true,\"deleteSensorReadouts\":true,\"columnNames\":[\"sample_column\"]}"

get_all_jobs

Retrieves a list of all queued and recently finished jobs.
Source code

GET 

http://localhost:8888/api/jobs/jobs

Return value

 Property name   Description                       Data type 
dqo_job_queue_initial_snapshot_model DqoJobQueueInitialSnapshotModel

Usage examples 

curl http://localhost:8888/api/jobs/jobs^
    -H "Accept: application/json"

get_job

Retrieves the current status of a single job, identified by a job id.
Source code

GET 

http://localhost:8888/api/jobs/jobs/{jobId}

Return value

 Property name   Description                       Data type 
dqo_job_history_entry_model DqoJobHistoryEntryModel

Parameters of this method are described below

 Property name   Description                       Data type   Required 
job_id Job id string

Usage examples 

curl http://localhost:8888/api/jobs/jobs/123123124324324^
    -H "Accept: application/json"

get_job_changes_since

Retrieves an incremental list of job changes (new jobs or job status changes)
Source code

GET 

http://localhost:8888/api/jobs/jobchangessince/{sequenceNumber}

Return value

 Property name   Description                       Data type 
dqo_job_queue_incremental_snapshot_model DqoJobQueueIncrementalSnapshotModel

Parameters of this method are described below

 Property name   Description                       Data type   Required 
sequence_number Change sequence number to get job changes after that sequence long

Usage examples 

curl http://localhost:8888/api/jobs/jobchangessince/3854372^
    -H "Accept: application/json"

import_tables

Starts a new background job that will import selected tables.
Source code

POST 

http://localhost:8888/api/jobs/importtables

Return value

 Property name   Description                       Data type 
import_tables_queue_job_result ImportTablesQueueJobResult

Parameters of this method are described below

 Property name   Description                       Data type   Required 
job_business_key Optional job business key that is a user assigned unique job id, used to check the job status by looking up the job by a user assigned identifier, instead of the DQOps assigned job identifier. string
wait Wait until the import tables job finishes to run, the default value is false (queue a background job and return the job id) boolean
wait_timeout The wait timeout in seconds, when the wait timeout elapses and the import tables job is still running, only the job id is returned without the results. The default timeout is 120 seconds, but could be reconfigured (see the 'dqo' cli command documentation). long

Request body

 Description                       Data type   Required 
Import tables job parameters ImportTablesQueueJobParameters

Usage examples 

curl -X POST http://localhost:8888/api/jobs/importtables^
    -H "Accept: application/json"^
    -H "Content-Type: application/json"^
    -d^
    "{\"connectionName\":\"sample_connection\",\"schemaName\":\"sample_schema\",\"tableNames\":[\"sample_table\"]}"

is_cron_scheduler_running

Checks if the DQOps internal CRON scheduler is running and processing jobs scheduled using cron expressions.
Source code

GET 

http://localhost:8888/api/jobs/scheduler/isrunning

Usage examples 

curl http://localhost:8888/api/jobs/scheduler/isrunning^
    -H "Accept: application/json"

run_checks

Starts a new background job that will run selected data quality checks
Source code

POST 

http://localhost:8888/api/jobs/runchecks

Return value

 Property name   Description                       Data type 
run_checks_queue_job_result RunChecksQueueJobResult

Parameters of this method are described below

 Property name   Description                       Data type   Required 
job_business_key Optional job business key that is a user assigned unique job id, used to check the job status by looking up the job by a user assigned identifier, instead of the DQOps assigned job identifier. string
wait Wait until the checks finish to run, the default value is false (queue a background job and return the job id) boolean
wait_timeout The wait timeout in seconds, when the wait timeout elapses and the checks are still running, only the job id is returned without the results. The default timeout is 120 seconds, but could be reconfigured (see the 'dqo' cli command documentation). long

Request body

 Description                       Data type   Required 
Data quality check run configuration (target checks and an optional time range) RunChecksParameters

Usage examples 

curl -X POST http://localhost:8888/api/jobs/runchecks^
    -H "Accept: application/json"^
    -H "Content-Type: application/json"^
    -d^
    "{\"check_search_filters\":{\"connection\":\"sample_connection\",\"fullTableName\":\"sample_schema.sample_table\",\"enabled\":true,\"column\":\"sample_column\",\"columnDataType\":\"string\"},\"dummy_execution\":false}"

start_cron_scheduler

Starts the job scheduler that runs monitoring jobs that are scheduled by assigning cron expressions.
Source code

POST 

http://localhost:8888/api/jobs/scheduler/status/start

Usage examples 

curl -X POST http://localhost:8888/api/jobs/scheduler/status/start^
    -H "Accept: application/json"

stop_cron_scheduler

Stops the job scheduler that runs monitoring jobs that are scheduled by assigning cron expressions.
Source code

POST 

http://localhost:8888/api/jobs/scheduler/status/stop

Usage examples 

curl -X POST http://localhost:8888/api/jobs/scheduler/status/stop^
    -H "Accept: application/json"

synchronize_folders

Starts multiple file synchronization jobs that will synchronize files from selected DQOps User home folders to the DQOps Cloud. The default synchronization mode is a full synchronization (upload local files, download new files from the cloud).
Source code

POST 

http://localhost:8888/api/jobs/synchronize

Return value

 Property name   Description                       Data type 
synchronize_multiple_folders_queue_job_result SynchronizeMultipleFoldersQueueJobResult

Parameters of this method are described below

 Property name   Description                       Data type   Required 
job_business_key Optional job business key that is a user assigned unique job id, used to check the job status by looking up the job by a user assigned identifier, instead of the DQOps assigned job identifier. string
wait Wait until the synchronize multiple folders job finishes to run, the default value is false (queue a background job and return the job id) boolean
wait_timeout The wait timeout in seconds, when the wait timeout elapses and the synchronization with the DQOps Cloud is still running, only the job id is returned without the results. The default timeout is 120 seconds, but could be reconfigured (see the 'dqo' cli command documentation). long

Request body

 Description                       Data type   Required 
Selection of folders that should be synchronized to the DQOps Cloud SynchronizeMultipleFoldersDqoQueueJobParameters

Usage examples 

curl -X POST http://localhost:8888/api/jobs/synchronize^
    -H "Accept: application/json"^
    -H "Content-Type: application/json"^
    -d^
    "{\"direction\":\"full\",\"forceRefreshNativeTables\":false,\"detectCronSchedules\":false,\"sources\":true,\"sensors\":true,\"rules\":true,\"checks\":true,\"settings\":true,\"credentials\":true,\"dataSensorReadouts\":true,\"dataCheckResults\":true,\"dataStatistics\":true,\"dataErrors\":true,\"dataIncidents\":true,\"synchronizeFolderWithLocalChanges\":false}"

wait_for_job

Waits for a job to finish. Returns the status of a finished job or a current state of a job that is still running, but the wait timeout elapsed.
Source code

GET 

http://localhost:8888/api/jobs/jobs/{jobId}/wait

Return value

 Property name   Description                       Data type 
dqo_job_history_entry_model DqoJobHistoryEntryModel

Parameters of this method are described below

 Property name   Description                       Data type   Required 
job_id Job id string
wait_timeout The wait timeout in seconds, when the wait timeout elapses and the job is still running, the method returns the job model that is not yet finished and has no results. The default timeout is 120 seconds, but could be reconfigured (see the 'dqo' cli command documentation). long

Usage examples 

curl http://localhost:8888/api/jobs/jobs/123123124324324/wait^
    -H "Accept: application/json"

wait_for_run_checks_job

Waits for a job to finish. Returns the status of a finished job or a current state of a job that is still running, but the wait timeout elapsed.
Source code

GET 

http://localhost:8888/api/jobs/runchecks/{jobId}/wait

Return value

 Property name   Description                       Data type 
run_checks_queue_job_result RunChecksQueueJobResult

Parameters of this method are described below

 Property name   Description                       Data type   Required 
job_id Job id, it can be a job business key assigned to the job or a job id generated by DQOps string
wait_timeout The wait timeout in seconds, when the wait timeout elapses and the job is still running, the method returns the job model that is not yet finished and has no results. The default timeout is 120 seconds, but could be reconfigured (see the 'dqo' cli command documentation). long

Usage examples 

curl http://localhost:8888/api/jobs/runchecks/123123124324324/wait^
    -H "Accept: application/json"