case class GetPercentileDFMetricCalculator(metricId: String, columns: Seq[String], accuracyError: Double, target: Double) extends PercentileDFCalculator with Product with Serializable
Calculates percentage value for given percentile value out of provided elements
Works for single column only!
- metricId
Id of the metric.
- columns
Sequence of columns which are used for metric calculation
- accuracyError
Required level of calculation accuracy
- target
Percentile value from set of column values.
- Alphabetic
- By Inheritance
- GetPercentileDFMetricCalculator
- Serializable
- Serializable
- Product
- Equals
- PercentileDFCalculator
- DFMetricCalculator
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
GetPercentileDFMetricCalculator(metricId: String, columns: Seq[String], accuracyError: Double, target: Double)
- metricId
Id of the metric.
- columns
Sequence of columns which are used for metric calculation
- accuracyError
Required level of calculation accuracy
- target
Percentile value from set of column values.
Value Members
-
val
accuracyError: Double
- Definition Classes
- GetPercentileDFMetricCalculator → PercentileDFCalculator
-
val
columns: Seq[String]
- Definition Classes
- GetPercentileDFMetricCalculator → DFMetricCalculator
-
def
errorMessage: String
Metric error message for cases when column value cannot be cast to number (double).
Metric error message for cases when column value cannot be cast to number (double).
- returns
Metric increment failure message.
- Definition Classes
- PercentileDFCalculator → DFMetricCalculator
-
def
errors(implicit errorDumpSize: Int, keyFields: Seq[String], colTypes: Map[String, DataType]): Column
Final metric errors aggregation expression.
Final metric errors aggregation expression. Collects all metric errors into an array column. The size of array is limited by maximum allowed error dump size parameter.
- errorDumpSize
Maximum allowed number of errors to be collected per single metric.
- keyFields
Sequence of source/stream key fields.
- colTypes
Map of column names to their datatype.
- returns
Spark expression that will yield array of metric errors.
- Definition Classes
- DFMetricCalculator
-
val
errorsCol: String
Name of the column that will store metric errors
Name of the column that will store metric errors
- Definition Classes
- DFMetricCalculator
-
val
isDirect: Boolean
- Definition Classes
- GetPercentileDFMetricCalculator → PercentileDFCalculator
-
val
metricId: String
Unlike RDD calculators, DF calculators are not groped by its type.
Unlike RDD calculators, DF calculators are not groped by its type. For each metric defined in DQ job, there will be created its own instance of DF calculator. Thus, DF metric calculators can be linked to metric definitions by metricId.
- Definition Classes
- GetPercentileDFMetricCalculator → DFMetricCalculator
-
val
metricName: MetricName
- Definition Classes
- GetPercentileDFMetricCalculator → DFMetricCalculator
-
def
result(implicit colTypes: Map[String, DataType]): Column
Final metric aggregation expression that MUST yield double value.
Final metric aggregation expression that MUST yield double value.
- colTypes
Map of column names to their datatype.
- returns
Spark expression that will yield double metric calculator result
- Definition Classes
- DFMetricCalculator
-
val
resultCol: String
Name of the column that will store metric result
Name of the column that will store metric result
- Definition Classes
- DFMetricCalculator
-
val
target: Double
- Definition Classes
- GetPercentileDFMetricCalculator → PercentileDFCalculator