c

org.checkita.dqf.core.metrics.df.regular.BasicNumericDFMetrics

GetPercentileDFMetricCalculator

case class GetPercentileDFMetricCalculator(metricId: String, columns: Seq[String], accuracyError: Double, target: Double) extends PercentileDFCalculator with Product with Serializable

Calculates percentage value for given percentile value out of provided elements

Works for single column only!

metricId

Id of the metric.

columns

Sequence of columns which are used for metric calculation

accuracyError

Required level of calculation accuracy

target

Percentile value from set of column values.

Linear Supertypes
Serializable, Serializable, Product, Equals, PercentileDFCalculator, DFMetricCalculator, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. GetPercentileDFMetricCalculator
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. PercentileDFCalculator
  7. DFMetricCalculator
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new GetPercentileDFMetricCalculator(metricId: String, columns: Seq[String], accuracyError: Double, target: Double)

    metricId

    Id of the metric.

    columns

    Sequence of columns which are used for metric calculation

    accuracyError

    Required level of calculation accuracy

    target

    Percentile value from set of column values.

Value Members

  1. val accuracyError: Double
  2. val columns: Seq[String]
  3. def errorMessage: String

    Metric error message for cases when column value cannot be cast to number (double).

    Metric error message for cases when column value cannot be cast to number (double).

    returns

    Metric increment failure message.

    Definition Classes
    PercentileDFCalculatorDFMetricCalculator
  4. def errors(implicit errorDumpSize: Int, keyFields: Seq[String], colTypes: Map[String, DataType]): Column

    Final metric errors aggregation expression.

    Final metric errors aggregation expression. Collects all metric errors into an array column. The size of array is limited by maximum allowed error dump size parameter.

    errorDumpSize

    Maximum allowed number of errors to be collected per single metric.

    keyFields

    Sequence of source/stream key fields.

    colTypes

    Map of column names to their datatype.

    returns

    Spark expression that will yield array of metric errors.

    Definition Classes
    DFMetricCalculator
  5. val errorsCol: String

    Name of the column that will store metric errors

    Name of the column that will store metric errors

    Definition Classes
    DFMetricCalculator
  6. val isDirect: Boolean
  7. val metricId: String

    Unlike RDD calculators, DF calculators are not groped by its type.

    Unlike RDD calculators, DF calculators are not groped by its type. For each metric defined in DQ job, there will be created its own instance of DF calculator. Thus, DF metric calculators can be linked to metric definitions by metricId.

    Definition Classes
    GetPercentileDFMetricCalculatorDFMetricCalculator
  8. val metricName: MetricName
  9. def result(implicit colTypes: Map[String, DataType]): Column

    Final metric aggregation expression that MUST yield double value.

    Final metric aggregation expression that MUST yield double value.

    colTypes

    Map of column names to their datatype.

    returns

    Spark expression that will yield double metric calculator result

    Definition Classes
    DFMetricCalculator
  10. val resultCol: String

    Name of the column that will store metric result

    Name of the column that will store metric result

    Definition Classes
    DFMetricCalculator
  11. val target: Double