case class CompletenessRDDMetricCalculator(nullCnt: Long, cellCnt: Long, includeEmptyStrings: Boolean, reversed: Boolean, failCount: Long = 0, status: CalculatorStatus = CalculatorStatus.Success, failMsg: String = "OK") extends RDDMetricCalculator with ReversibleRDDCalculator with Product with Serializable
Calculates completeness of values in the specified columns
- nullCnt
Current amount of null values.
- cellCnt
Current amount of cells.
- includeEmptyStrings
Flag which sets whether empty strings are considered in addition to null values.
- reversed
Boolean flag indicating whether error collection logic should be direct or reversed.
- returns
result map with keys: "COMPLETENESS"
- Alphabetic
- By Inheritance
- CompletenessRDDMetricCalculator
- Serializable
- Serializable
- Product
- Equals
- ReversibleRDDCalculator
- RDDMetricCalculator
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
- new CompletenessRDDMetricCalculator(includeEmptyStrings: Boolean, reversed: Boolean)
-
new
CompletenessRDDMetricCalculator(nullCnt: Long, cellCnt: Long, includeEmptyStrings: Boolean, reversed: Boolean, failCount: Long = 0, status: CalculatorStatus = CalculatorStatus.Success, failMsg: String = "OK")
- nullCnt
Current amount of null values.
- cellCnt
Current amount of cells.
- includeEmptyStrings
Flag which sets whether empty strings are considered in addition to null values.
- reversed
Boolean flag indicating whether error collection logic should be direct or reversed.
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
- val cellCnt: Long
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
def
copyWithError(status: CalculatorStatus, msg: String, failInc: Long = 1): RDDMetricCalculator
Copy calculator with error status and corresponding message.
Copy calculator with error status and corresponding message.
- status
Calculator status to copy with
- msg
Failure message
- failInc
Failure increment
- returns
Copy of this calculator with error status
- Attributes
- protected
- Definition Classes
- CompletenessRDDMetricCalculator → RDDMetricCalculator
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
val
failCount: Long
- Attributes
- protected
- Definition Classes
- CompletenessRDDMetricCalculator → RDDMetricCalculator
-
val
failMsg: String
- Attributes
- protected
- Definition Classes
- CompletenessRDDMetricCalculator → RDDMetricCalculator
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
getFailCounter: Long
Gets current metric failure counts
-
def
getFailMessage: String
Gets current failure or error message
-
def
getStatus: CalculatorStatus
Gets current metric calculator status
- val includeEmptyStrings: Boolean
-
def
increment(values: Seq[Any]): RDDMetricCalculator
Safely updates metric calculator with respect to specified error collection logic (direct or reversed).
Safely updates metric calculator with respect to specified error collection logic (direct or reversed).
- values
values to process
- returns
updated calculator
- Definition Classes
- ReversibleRDDCalculator
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
merge(m2: RDDMetricCalculator): RDDMetricCalculator
Merges two metric calculators together
Merges two metric calculators together
- m2
second metric calculator
- returns
merged metric calculator
- Definition Classes
- CompletenessRDDMetricCalculator → RDDMetricCalculator
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- val nullCnt: Long
-
def
result(): Map[String, (Double, Option[String])]
Gets results of calculator in the current state
Gets results of calculator in the current state
- returns
Map of (result_name -> (result, additionalResults))
- Definition Classes
- CompletenessRDDMetricCalculator → RDDMetricCalculator
-
val
reversed: Boolean
- Attributes
- protected
- Definition Classes
- CompletenessRDDMetricCalculator → ReversibleRDDCalculator
-
val
status: CalculatorStatus
- Attributes
- protected
- Definition Classes
- CompletenessRDDMetricCalculator → RDDMetricCalculator
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
tryToIncrement(values: Seq[Any]): RDDMetricCalculator
Increment metric calculator.
Increment metric calculator. May throw an exception. Direct error collection logic implies that any non-null (or non-empty if
includeEmptyStringsistrue) values are considered as metric failure and are collected.- values
values to process
- returns
updated calculator or throws an exception
- Attributes
- protected
- Definition Classes
- CompletenessRDDMetricCalculator → RDDMetricCalculator
-
def
tryToIncrementReversed(values: Seq[Any]): RDDMetricCalculator
Increment metric calculator with REVERSED error collection logic.
Increment metric calculator with REVERSED error collection logic. May throw an exception.
- values
values to process
- returns
updated calculator or throws an exception
- Attributes
- protected
- Definition Classes
- CompletenessRDDMetricCalculator → ReversibleRDDCalculator
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()