Packages

o

org.checkita.dqf.core.metrics.rdd.regular

MultiColumnRDDMetrics

object MultiColumnRDDMetrics

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. MultiColumnRDDMetrics
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. case class ColumnEqRDDMetricCalculator(cnt: Int, reversed: Boolean, failCount: Long = 0, status: CalculatorStatus = CalculatorStatus.Success, failMsg: String = "OK") extends RDDMetricCalculator with ReversibleRDDCalculator with Product with Serializable

    Calculates amount rows where elements in the given columns are equal

    Calculates amount rows where elements in the given columns are equal

    cnt

    current counter

    returns

    result map with keys: "COLUMN_EQ"

  2. case class CovarianceRDDMetricCalculator(lMean: Double, rMean: Double, coMoment: Double, n: Long, failCount: Long = 0, status: CalculatorStatus = CalculatorStatus.Success, failMsg: String = "OK") extends RDDMetricCalculator with Product with Serializable

    Calculates covariance between values of two columns

    Calculates covariance between values of two columns

    lMean

    Mean of the first column

    rMean

    Mean of the second column

    coMoment

    current co-moment

    n

    number of records (Idea here is that in order to correctly calculate covariance of the two columns, all cells' values must be convertable to Double. Otherwise the metric result won't be reliable, and, NaN value will be returned instead)

    returns

    result map with keys:

    • "CO_MOMENT"
    • "COVARIANCE"
    • "COVARIANCE_BESSEL"
  3. case class DayDistanceRDDMetricCalculator(cnt: Double, dateFormat: String, threshold: Int, reversed: Boolean, failCount: Long = 0, status: CalculatorStatus = CalculatorStatus.Success, failMsg: String = "OK") extends RDDMetricCalculator with ReversibleRDDCalculator with Product with Serializable

    Calculates the number of the rows for which the day difference between two columns given as input is less than the threshold (number of days)

    Calculates the number of the rows for which the day difference between two columns given as input is less than the threshold (number of days)

    cnt

    Current coutner

    dateFormat

    Date format for values in columns

    threshold

    Maximum allowed day distance between dates in columns

    returns

    result map with keys: "DAY_DISTANCE"

  4. case class LevenshteinDistanceRDDMetricCalculator(cnt: Double, threshold: Double, normalize: Boolean, reversed: Boolean, failCount: Long = 0, status: CalculatorStatus = CalculatorStatus.Success, failMsg: String = "OK") extends RDDMetricCalculator with ReversibleRDDCalculator with Product with Serializable

    Calculates amount of rows where Levenshtein distance between 2 columns is less than threshold.

    Calculates amount of rows where Levenshtein distance between 2 columns is less than threshold.

    cnt

    current success counter

    threshold

    Threshold (should be within [0, 1] range for normalized results)

    normalize

    Flag to define whether distance should be normalized over maximum length of two input strings

    returns

    result map with keys: "LEVENSHTEIN_DISTANCE"

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  16. def toString(): String
    Definition Classes
    AnyRef → Any
  17. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  18. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  19. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped