Packages

object PostValidation

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. PostValidation
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. final case class Field(name: String, value: Any, path: String) extends Product with Serializable

    Case class to represent configuration field

    Case class to represent configuration field

    name

    Name of the configuration field

    value

    Value of the configuration field

    path

    Path to a configuration field from the root

  2. final class Reference extends AnyRef

    Class to represent cross reference used in configuration.

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. val allPostValidations: Seq[(ConfigObject) ⇒ Vector[ConfigReaderFailure]]

    All post validations for Data Quality job-level configuration

  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  10. def findCycles[N](graph: Map[N, List[N]]): List[List[N]]

    Tail recursive DFS to find cycles in directed graph stored in adjacent list format: Map(node -> List(adjacent node))

    Tail recursive DFS to find cycles in directed graph stored in adjacent list format: Map(node -> List(adjacent node))

    N

    Type of the graph nodes.

    graph

    Graph to traverse for cycles.

    returns

    List of cyclic paths in the graph.

  11. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  12. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  13. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  14. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  18. def toString(): String
    Definition Classes
    AnyRef → Any
  19. val validateArimaTrendMetricOrder: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Ensure validity of ARIMA model order parameters in ARIMA trend metric configuration.

    Ensure validity of ARIMA model order parameters in ARIMA trend metric configuration.

    Note

    This check is run at post-validation stage, as it is quite difficult to derive pureconfig readers for sealed trait families (thus approach is used to define kinded configurations) to run check during config parsing.

  20. val validateBatchOrStream: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains both batch and streaming sources defined.

  21. val validateComposedMetrics: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains errors in composed metric formulas: wrong equation that cannot be parsed

  22. val validateExpressionCheckRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains errors in expression check formulas:

    Validation to check if DQ job configuration contains errors in expression check formulas:

    • missing reference to metrics
    • wrong equation that cannot be parsed
  23. val validateGreenplumSourceRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from table sources to pivotal connections

  24. val validateIds: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains duplicate IDs.

  25. val validateJdbcSourceRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from table sources to jdbc connections

  26. val validateKafkaSourceRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from kafka topics sources to kafka connections

  27. val validateLoadCheckRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from load checks to sources

    Validation to check if DQ job configuration contains missing references from load checks to sources

    Note

    Load checks are not applied to streaming sources

  28. val validateLoadCheckSchemaRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from schema match load checks to schemas

  29. val validateMetricCrossReferences: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if composed and trend metrics definitions does not create reference cycles.

    Validation to check if composed and trend metrics definitions does not create reference cycles. Metric computation must be acyclic.

  30. val validateRegularMetricRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from regular metrics to sources

  31. val validateSnapshotCheckRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from snapshot checks to metrics

  32. val validateSourceSchemaRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from sources to schemas

  33. val validateTargetCheckRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from targets to checks

  34. val validateTargetConnectionRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from targets to connections

  35. val validateTargetMetricRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from targets to metrics

  36. val validateTrendCheckRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from trend checks to metrics

  37. val validateTrendMetricWindowSettings: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Ensure that windowSize and windowOffset parameters in trend metrics do comply with rule:

    Ensure that windowSize and windowOffset parameters in trend metrics do comply with rule:

    • if rule is record then windowSize and windowOffset must be non-negative integers
    • if rule is datetime then windowSize and windowOffset must be valid non-negative durations
    Note

    This check is run at post-validation stage, as it is quite difficult to derive pureconfig readers for sealed trait families (thus approach is used to define kinded configurations) to run check during config parsing.

  38. val validateVirtualSourceRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains missing references from virtual sources to already defined sources.

    Validation to check if DQ job configuration contains missing references from virtual sources to already defined sources. Check is recursive: virtual sources can also refer to other virtual sources defined above.

    Note

    batch and streaming sources are checked separately.

  39. val validateVirtualStreams: (ConfigObject) ⇒ Vector[ConfigReaderFailure]

    Validation to check if DQ job configuration contains non-streamable kinds of virtual sources defined in virtualStreams list.

  40. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  41. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  42. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped