c

org.checkita.dqf.core.metrics.df.regular.BasicNumericDFMetrics

NumberInDomainDFMetricCalculator

case class NumberInDomainDFMetricCalculator(metricId: String, columns: Seq[String], domain: Set[Double], reversed: Boolean) extends ConditionalDFCalculator with Product with Serializable

Calculated amount of numbers in provided domain set

metricId

Id of the metric.

columns

Sequence of columns which are used for metric calculation

domain

Set of numbers that represents the requested domain

reversed

Boolean flag indicating whether error collection logic should be reversed for this metric

Linear Supertypes
Serializable, Serializable, Product, Equals, ConditionalDFCalculator, ReversibleDFCalculator, DFMetricCalculator, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. NumberInDomainDFMetricCalculator
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. ConditionalDFCalculator
  7. ReversibleDFCalculator
  8. DFMetricCalculator
  9. AnyRef
  10. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new NumberInDomainDFMetricCalculator(metricId: String, columns: Seq[String], domain: Set[Double], reversed: Boolean)

    metricId

    Id of the metric.

    columns

    Sequence of columns which are used for metric calculation

    domain

    Set of numbers that represents the requested domain

    reversed

    Boolean flag indicating whether error collection logic should be reversed for this metric

Value Members

  1. val columns: Seq[String]
  2. val domain: Set[Double]
  3. def errorMessage: String

    For direct error collection logic numeric values which are outside of provided domain are considered as metric failure.

    For direct error collection logic numeric values which are outside of provided domain are considered as metric failure. For reversed error collection logic numeric values which are within provided domain are considered as metric failure.

    returns

    Metric increment failure message.

    Definition Classes
    NumberInDomainDFMetricCalculatorDFMetricCalculator
  4. def errors(implicit errorDumpSize: Int, keyFields: Seq[String], colTypes: Map[String, DataType]): Column

    Final metric errors aggregation expression.

    Final metric errors aggregation expression. Collects all metric errors into an array column. The size of array is limited by maximum allowed error dump size parameter.

    errorDumpSize

    Maximum allowed number of errors to be collected per single metric.

    keyFields

    Sequence of source/stream key fields.

    colTypes

    Map of column names to their datatype.

    returns

    Spark expression that will yield array of metric errors.

    Definition Classes
    DFMetricCalculator
  5. val errorsCol: String

    Name of the column that will store metric errors

    Name of the column that will store metric errors

    Definition Classes
    DFMetricCalculator
  6. val metricId: String

    Unlike RDD calculators, DF calculators are not groped by its type.

    Unlike RDD calculators, DF calculators are not groped by its type. For each metric defined in DQ job, there will be created its own instance of DF calculator. Thus, DF metric calculators can be linked to metric definitions by metricId.

    Definition Classes
    NumberInDomainDFMetricCalculatorDFMetricCalculator
  7. val metricName: MetricName
  8. def result(implicit colTypes: Map[String, DataType]): Column

    Final metric aggregation expression that MUST yield double value.

    Final metric aggregation expression that MUST yield double value.

    colTypes

    Map of column names to their datatype.

    returns

    Spark expression that will yield double metric calculator result

    Definition Classes
    DFMetricCalculator
  9. val resultCol: String

    Name of the column that will store metric result

    Name of the column that will store metric result

    Definition Classes
    DFMetricCalculator