Packages

t

org.checkita.dqf.targets.builders.dataframe

ErrorDataFrameBuilder

trait ErrorDataFrameBuilder[T <: ErrorCollTargetConfig] extends TargetBuilder[T, DataFrame] with BuildHelpers

Linear Supertypes
BuildHelpers, TargetBuilder[T, DataFrame], AnyRef, Any
Known Subclasses
ErrorCollFileTargetProcessor, ErrorCollHiveTargetProcessor
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ErrorDataFrameBuilder
  2. BuildHelpers
  3. TargetBuilder
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. def build(target: T, results: ResultSet)(implicit settings: AppSettings, spark: SparkSession): Result[DataFrame]

    Build target output given the target configuration

    Build target output given the target configuration

    target

    Target configuration

    results

    All job results

    settings

    Implicit application settings object

    spark

    Implicit spark session object

    returns

    Target result in form of Spark DataFrame

    Definition Classes
    ErrorDataFrameBuilder → TargetBuilder
  2. def filterErrors(errors: Seq[ResultMetricError], requested: Seq[String], dumpSize: Int): Seq[ResultMetricError]

    Filters errors to be send in target.

    Filters errors to be send in target.

    errors

    Sequence of all collected metric errors

    requested

    Sequence of metric IDs for which metric errors are requested

    dumpSize

    Maximum number of errors to be collected per each metric

    returns

    Filtered sequence of metric errors

    Definition Classes
    BuildHelpers
    Note

    If requested metric IDs are not provided (sequence is empty) then errors for all metrics are returned.

  3. def readTemplate(uri: String): Option[String]

    Safely reads template from file

    Safely reads template from file

    uri

    Path to a template file

    returns

    Template string read from path or None if read operation wasn't successful.

    Definition Classes
    BuildHelpers