c

org.checkita.dqf.core.metrics.rdd.regular.BasicStringRDDMetrics

DistinctValuesRDDMetricCalculator

case class DistinctValuesRDDMetricCalculator(uniqueValues: Set[String] = Set.empty[String], failCount: Long = 0, status: CalculatorStatus = CalculatorStatus.Success, failMsg: String = "OK") extends RDDMetricCalculator with Product with Serializable

Calculates count of distinct values in processed elements WARNING: Uses set without any kind of trimming and hashing. Returns the exact count. So if a big diversion of elements needs to be processed and exact result is not mandatory, then it's better to use HyperLogLog version called "APPROXIMATE_DISTINCT_VALUES".

uniqueValues

Set of processed values

returns

result map with keys: "DISTINCT_VALUES"

Linear Supertypes
Serializable, Serializable, Product, Equals, RDDMetricCalculator, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DistinctValuesRDDMetricCalculator
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. RDDMetricCalculator
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new DistinctValuesRDDMetricCalculator()
  2. new DistinctValuesRDDMetricCalculator(uniqueValues: Set[String] = Set.empty[String], failCount: Long = 0, status: CalculatorStatus = CalculatorStatus.Success, failMsg: String = "OK")

    uniqueValues

    Set of processed values

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. def copyWithError(status: CalculatorStatus, msg: String, failInc: Long = 1): RDDMetricCalculator

    Copy calculator with error status and corresponding message.

    Copy calculator with error status and corresponding message.

    status

    Calculator status to copy with

    msg

    Failure message

    failInc

    Failure increment

    returns

    Copy of this calculator with error status

    Attributes
    protected
    Definition Classes
    DistinctValuesRDDMetricCalculator → RDDMetricCalculator
  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. val failCount: Long
    Attributes
    protected
    Definition Classes
    DistinctValuesRDDMetricCalculator → RDDMetricCalculator
  9. val failMsg: String
    Attributes
    protected
    Definition Classes
    DistinctValuesRDDMetricCalculator → RDDMetricCalculator
  10. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  11. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  12. def getFailCounter: Long

    Gets current metric failure counts

    Gets current metric failure counts

    returns

    Failure count

    Definition Classes
    RDDMetricCalculator
  13. def getFailMessage: String

    Gets current failure or error message

    Gets current failure or error message

    returns

    Failure message

    Definition Classes
    RDDMetricCalculator
  14. def getStatus: CalculatorStatus

    Gets current metric calculator status

    Gets current metric calculator status

    returns

    Calculator status

    Definition Classes
    RDDMetricCalculator
  15. def increment(values: Seq[Any]): RDDMetricCalculator

    Safely updates metric calculator

    Safely updates metric calculator

    values

    values to process

    returns

    updated calculator

    Definition Classes
    RDDMetricCalculator
  16. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  17. def merge(m2: RDDMetricCalculator): RDDMetricCalculator

    Merges two metric calculators together

    Merges two metric calculators together

    m2

    second metric calculator

    returns

    merged metric calculator

    Definition Classes
    DistinctValuesRDDMetricCalculator → RDDMetricCalculator
  18. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  19. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  20. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  21. def result(): Map[String, (Double, Option[String])]

    Gets results of calculator in the current state

    Gets results of calculator in the current state

    returns

    Map of (result_name -> (result, additionalResults))

    Definition Classes
    DistinctValuesRDDMetricCalculator → RDDMetricCalculator
  22. val status: CalculatorStatus
    Attributes
    protected
    Definition Classes
    DistinctValuesRDDMetricCalculator → RDDMetricCalculator
  23. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  24. def tryToIncrement(values: Seq[Any]): RDDMetricCalculator

    Increment metric calculator.

    Increment metric calculator. May throw an exception.

    values

    values to process

    returns

    updated calculator or throws an exception

    Attributes
    protected
    Definition Classes
    DistinctValuesRDDMetricCalculator → RDDMetricCalculator
  25. val uniqueValues: Set[String]
  26. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from RDDMetricCalculator

Inherited from AnyRef

Inherited from Any

Ungrouped