Type Members

1.  final case class CheckAlertEmailTargetConfig(id: ID, recipients: Refined[Seq[Email], NonEmpty], checks: Seq[NonEmptyString] = Seq.empty, subjectTemplate: Option[NonEmptyString], template: Option[NonEmptyString], templateFile: Option[URI]) extends CheckAlertTargetConfig with EmailOutputConfig with Product with Serializable

   Check alert target configuration with output to recipients via email

   Check alert target configuration with output to recipients via email

   id

   Check alert ID

   recipients

   Non-empty sequence of recipients' emails.

   checks

   Sequence of checks to send alerts for. Default: empty sequence (send alerts for all checks)

   subjectTemplate

   Mustache template used to customize email subject. If omitted, default subject name is used.

   template

   Mustache Html template for email body.

   templateFile

   Location of file with Mustache Html template for email body.

   Note

   Template for email body can be provided either explicitly in template argument or read from file provided in templateFile argument. Both of these arguments are allowed. Also, if both of these arguments are omitted, then default email body is used.

2.  final case class CheckAlertMattermostTargetConfig(id: ID, recipients: Refined[Seq[MMRecipient], NonEmpty], checks: Seq[NonEmptyString] = Seq.empty, template: Option[NonEmptyString], templateFile: Option[URI]) extends CheckAlertTargetConfig with MattermostOutputConfig with Product with Serializable

   Check alert target configuration with output to recipients via Mattermost

   Check alert target configuration with output to recipients via Mattermost

   id

   Check alert ID

   recipients

   Non-empty sequence of mattermost channel names or usernames

   checks

   Sequence of checks to send alerts for. Default: empty sequence (send alerts for all checks)

3.  sealed abstract class CheckAlertTargetConfig extends TargetConfig

   Base class for all check alert target configurations.

   Base class for all check alert target configurations. Check alert targets must have an ID and sequence of checks for which the alerts are sent.

4.  final case class CheckAlertTargetsConfig(email: Seq[CheckAlertEmailTargetConfig] = Seq.empty, mattermost: Seq[CheckAlertMattermostTargetConfig] = Seq.empty) extends Product with Serializable

   Data Quality job configuration section describing check alert targets

   Data Quality job configuration section describing check alert targets

   email

   Sequence of configurations for check alert targets sent to recipients via email

   mattermost

   Sequence of configurations for check alert targets sent to recipients via Mattermost

5.  final case class ErrorCollFileTargetConfig(metrics: Seq[NonEmptyString] = Seq.empty, dumpSize: Option[PositiveInt], save: FileOutputConfig) extends ErrorCollTargetConfig with SaveToFileConfig with Product with Serializable

   Error collection target configuration that is written to a file

6.  final case class ErrorCollHiveTargetConfig(schema: NonEmptyString, table: NonEmptyString, metrics: Seq[NonEmptyString] = Seq.empty, dumpSize: Option[PositiveInt]) extends ErrorCollTargetConfig with HiveOutputConfig with Product with Serializable

   Error collection target configuration with output to Hive table.

   Error collection target configuration with output to Hive table.

   schema

   Hive schema to write into

   table

   Hive table to write into

   metrics

   Sequence of metrics to collect errors for. Default: empty sequence (collect errors for all metrics)

   dumpSize

   Maximum number of errors collected per each metric.

7.  final case class ErrorCollKafkaTargetConfig(connection: NonEmptyString, topic: NonEmptyString, options: Seq[SparkParam] = Seq.empty, metrics: Seq[NonEmptyString] = Seq.empty, dumpSize: Option[PositiveInt]) extends ErrorCollTargetConfig with KafkaOutputConfig with Product with Serializable

   Error collection target configuration with output to a Kafka topic.

   Error collection target configuration with output to a Kafka topic.

   connection

   reference to Kafka connection ID

   topic

   Kafka topic name to write into

   options

   Sequence of additional Kafka options

   metrics

   Sequence of metrics to collect errors for. Default: empty sequence (collect errors for all metrics)

   dumpSize

   Maximum number of errors collected per each metric. Default: 100

8.  sealed abstract class ErrorCollTargetConfig extends TargetConfig

   Base class for all error collection target configurations.

   Base class for all error collection target configurations. All error collection targets must have a sequence of metrics for which errors are collected as well as maximum number of errors to be dumped per each metric (both could be optional with default values).

9.  final case class ErrorCollTargetsConfig(file: Option[ErrorCollFileTargetConfig], hive: Option[ErrorCollHiveTargetConfig], kafka: Option[ErrorCollKafkaTargetConfig]) extends Product with Serializable

   Data Quality job configuration section describing error collection targets

   Data Quality job configuration section describing error collection targets

   file

   Optional configuration for error collection targets output to a file

   hive

   Optional configuration for error collection targets output to Hive

   kafka

   Optional configuration for error collection targets output to a Kafka topic

10.  final case class ResultFileTargetConfig(resultTypes: Refined[Seq[ResultTargetType], NonEmpty], save: FileOutputConfig) extends ResultTargetConfig with SaveToFileConfig with Product with Serializable

    Result target configuration that is written to a file.

11.  final case class ResultHiveTargetConfig(resultTypes: Refined[Seq[ResultTargetType], NonEmpty], schema: NonEmptyString, table: NonEmptyString) extends ResultTargetConfig with HiveOutputConfig with Product with Serializable

    Result target configuration with output to Hive table.

    Result target configuration with output to Hive table.

    resultTypes

    Non-empty sequence of result types to save

    schema

    Hive schema to write into

    table

    Hive table to write into

12.  final case class ResultKafkaTargetConfig(resultTypes: Refined[Seq[ResultTargetType], NonEmpty], connection: NonEmptyString, topic: NonEmptyString, options: Seq[SparkParam] = Seq.empty) extends ResultTargetConfig with KafkaOutputConfig with Product with Serializable

    Result target configuration with output to Kafka topic

    Result target configuration with output to Kafka topic

    resultTypes

    Non-empty sequence of result types to save

    connection

    reference to Kafka connection ID

    topic

    Kafka topic name to write into

    options

    Sequence of additional Kafka options

13.  sealed abstract class ResultTargetConfig extends TargetConfig

    Base class for all result target configurations.

    Base class for all result target configurations. All result targets must have non-empty sequence of result types to save.

14.  final case class ResultTargetsConfig(file: Option[ResultFileTargetConfig], hive: Option[ResultHiveTargetConfig], kafka: Option[ResultKafkaTargetConfig]) extends Product with Serializable

    Data Quality job configuration section describing result targets.

    Data Quality job configuration section describing result targets.

    file

    Optional configuration for result targets output to a file

    hive

    Optional configuration for result targets output to Hive

    kafka

    Optional configuration for result targets output to a Kafka topic

15.  final case class SummaryEmailTargetConfig(recipients: Refined[Seq[Email], NonEmpty], attachMetricErrors: Boolean = false, attachFailedChecks: Boolean = false, metrics: Seq[NonEmptyString] = Seq.empty, dumpSize: Option[PositiveInt], subjectTemplate: Option[NonEmptyString], template: Option[NonEmptyString], templateFile: Option[URI]) extends SummaryTargetConfig with EmailOutputConfig with Product with Serializable

    Summary target configuration with output to recipients via email.

    Summary target configuration with output to recipients via email.

    recipients

    Non-empty sequence of recipients' emails.

    attachMetricErrors

    Boolean flag indicating whether to attach metric error collection report to summary email. Default: false

    metrics

    Sequence of metrics to collect errors for. Default: empty sequence (collect errors for all metrics)

    dumpSize

    Maximum number of errors collected per each metric. Default: 100

    subjectTemplate

    Mustache template used to customize email subject. If omitted, default subject name is used.

    template

    Mustache Html template for email body.

    templateFile

    Location of file with Mustache Html template for email body.

    Note

    Template for email body can be provided either explicitly in template argument or read from file provided in templateFile argument. Both of these arguments are allowed. Also, if both of these arguments are omitted, then default email body is used.

16.  final case class SummaryKafkaTargetConfig(connection: NonEmptyString, topic: NonEmptyString, options: Seq[SparkParam] = Seq.empty) extends SummaryTargetConfig with KafkaOutputConfig with Product with Serializable

    Summary target configuration with output to a Kafka topic.

    Summary target configuration with output to a Kafka topic.

    connection

    reference to Kafka connection ID

    topic

    Kafka topic name to write into

    options

    Sequence of additional Kafka options

17.  final case class SummaryMattermostTargetConfig(recipients: Refined[Seq[MMRecipient], NonEmpty], attachMetricErrors: Boolean = false, attachFailedChecks: Boolean = false, metrics: Seq[NonEmptyString] = Seq.empty, dumpSize: Option[PositiveInt], template: Option[NonEmptyString], templateFile: Option[URI]) extends SummaryTargetConfig with MattermostOutputConfig with Product with Serializable

    Summary target configuration with output to recipients via Mattermost

    Summary target configuration with output to recipients via Mattermost

    recipients

    Non-empty sequence of mattermost channel names or usernames

    attachMetricErrors

    Boolean flag indicating whether to attach metric error collection report to summary email. Default: false

    metrics

    Sequence of metrics to collect errors for. Default: empty sequence (collect errors for all metrics)

    dumpSize

    Maximum number of errors collected per each metric. Default: 100

18.  sealed abstract class SummaryTargetConfig extends TargetConfig

    Base class for all summary target configurations.

    Base class for all summary target configurations. Some of output channels support attachments to the messages. Therefore, it is possible to indicate whether metric errors or failed check attachments are required. In addition, for metric errors attachment it is possible to narrow list of metrics by providing explicit list of IDs and also to limit number of errors per each metric by setting dumpSize.

19.  final case class SummaryTargetsConfig(email: Option[SummaryEmailTargetConfig], mattermost: Option[SummaryMattermostTargetConfig], kafka: Option[SummaryKafkaTargetConfig]) extends Product with Serializable

    Data Quality job configuration section describing summary targets

    Data Quality job configuration section describing summary targets

    email

    Optional configuration for summary target reports sent to recipients via email

    mattermost

    Optional configuration for summary target reports sent to recipients via mattermost

    kafka

    Optional configuration for summary target reports sent to a Kafka topic

20.  sealed abstract class TargetConfig extends AnyRef

    Base class for all target configurations

21.  final case class TargetsConfig(results: Option[ResultTargetsConfig], errorCollection: Option[ErrorCollTargetsConfig], summary: Option[SummaryTargetsConfig], checkAlerts: Option[CheckAlertTargetsConfig]) extends Product with Serializable

    Data Quality job configuration section describing all targets

    Data Quality job configuration section describing all targets

    results

    Result targets of all subtypes

    errorCollection

    Error collection targets of all subtypes

    summary

    Summary targets of all subtypes

    checkAlerts

    Check alert targets of all subtypes