object PostValidation
- Alphabetic
- By Inheritance
- PostValidation
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Type Members
-
final
case class
Field(name: String, value: Any, path: String) extends Product with Serializable
Case class to represent configuration field
Case class to represent configuration field
- name
Name of the configuration field
- value
Value of the configuration field
- path
Path to a configuration field from the root
-
final
class
Reference extends AnyRef
Class to represent cross reference used in configuration.
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
val
allPostValidations: Seq[(ConfigObject) ⇒ Vector[ConfigReaderFailure]]
All post validations for Data Quality job-level configuration
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
def
findCycles[N](graph: Map[N, List[N]]): List[List[N]]
Tail recursive DFS to find cycles in directed graph stored in adjacent list format: Map(node -> List(adjacent node))
Tail recursive DFS to find cycles in directed graph stored in adjacent list format: Map(node -> List(adjacent node))
- N
Type of the graph nodes.
- graph
Graph to traverse for cycles.
- returns
List of cyclic paths in the graph.
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
val
validateArimaTrendMetricOrder: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Ensure validity of ARIMA model order parameters in ARIMA trend metric configuration.
Ensure validity of ARIMA model order parameters in ARIMA trend metric configuration.
- Note
This check is run at post-validation stage, as it is quite difficult to derive pureconfig readers for sealed trait families (thus approach is used to define kinded configurations) to run check during config parsing.
-
val
validateBatchOrStream: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains both batch and streaming sources defined.
-
val
validateComposedMetrics: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains errors in composed metric formulas: wrong equation that cannot be parsed
-
val
validateExpressionCheckRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains errors in expression check formulas:
Validation to check if DQ job configuration contains errors in expression check formulas:
- missing reference to metrics
- wrong equation that cannot be parsed
-
val
validateGreenplumSourceRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from table sources to pivotal connections
-
val
validateIds: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains duplicate IDs.
-
val
validateJdbcSourceRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from table sources to jdbc connections
-
val
validateKafkaSourceRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from kafka topics sources to kafka connections
-
val
validateLoadCheckRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from load checks to sources
Validation to check if DQ job configuration contains missing references from load checks to sources
- Note
Load checks are not applied to streaming sources
-
val
validateLoadCheckSchemaRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from schema match load checks to schemas
-
val
validateMetricCrossReferences: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if composed and trend metrics definitions does not create reference cycles.
Validation to check if composed and trend metrics definitions does not create reference cycles. Metric computation must be acyclic.
-
val
validateRegularMetricRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from regular metrics to sources
-
val
validateSnapshotCheckRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from snapshot checks to metrics
-
val
validateSourceSchemaRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from sources to schemas
-
val
validateTargetCheckRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from targets to checks
-
val
validateTargetConnectionRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from targets to connections
-
val
validateTargetMetricRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from targets to metrics
-
val
validateTrendCheckRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from trend checks to metrics
-
val
validateTrendMetricWindowSettings: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Ensure that windowSize and windowOffset parameters in trend metrics do comply with rule:
Ensure that windowSize and windowOffset parameters in trend metrics do comply with rule:
- if rule is record then windowSize and windowOffset must be non-negative integers
- if rule is datetime then windowSize and windowOffset must be valid non-negative durations
- Note
This check is run at post-validation stage, as it is quite difficult to derive pureconfig readers for sealed trait families (thus approach is used to define kinded configurations) to run check during config parsing.
-
val
validateVirtualSourceRefs: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains missing references from virtual sources to already defined sources.
Validation to check if DQ job configuration contains missing references from virtual sources to already defined sources. Check is recursive: virtual sources can also refer to other virtual sources defined above.
- Note
batch and streaming sources are checked separately.
-
val
validateVirtualStreams: (ConfigObject) ⇒ Vector[ConfigReaderFailure]
Validation to check if DQ job configuration contains non-streamable kinds of virtual sources defined in virtualStreams list.
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()