Packages

object LoadChecks

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. LoadChecks
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. final case class ColumnsExistsLoadCheckConfig(id: ID, description: Option[NonEmptyString], source: NonEmptyString, columns: NonEmptyStringSeq, isCritical: Boolean = false, metadata: Seq[SparkParam] = Seq.empty) extends LoadCheckConfig with Product with Serializable

    Columns exists load check configuration

    Columns exists load check configuration

    id

    Load check ID

    description

    Load check description

    source

    Source ID to be checked

    columns

    Sequence of columns that must be presented in the source

    isCritical

    Flag if check is critical

    metadata

    List of metadata parameters specific to this load check

  2. final case class ExactColNumLoadCheckConfig(id: ID, description: Option[NonEmptyString], source: NonEmptyString, option: PositiveInt, isCritical: Boolean = false, metadata: Seq[SparkParam] = Seq.empty) extends LoadCheckConfig with Product with Serializable

    Exact column number load check configuration

    Exact column number load check configuration

    id

    Load check ID

    description

    Load check description

    source

    Source ID to be checked

    option

    Required number of columns

    isCritical

    Flag if check is critical

    metadata

    List of metadata parameters specific to this load check

  3. sealed abstract class LoadCheckConfig extends JobConfigEntity

    Base class for all load checks configurations.

    Base class for all load checks configurations. All load checks are described as DQ entities that must have a reference to source ID which is checked. Additionally, it is required to define method for appropriate load check calculator initiation.

  4. final case class LoadChecksConfig(exactColumnNum: Seq[ExactColNumLoadCheckConfig] = Seq.empty, minColumnNum: Seq[MinColNumLoadCheckConfig] = Seq.empty, columnsExist: Seq[ColumnsExistsLoadCheckConfig] = Seq.empty, schemaMatch: Seq[SchemaMatchLoadCheckConfig] = Seq.empty) extends Product with Serializable

    Data Quality job configuration section describing load checks

    Data Quality job configuration section describing load checks

    exactColumnNum

    Sequence of load checks for exact column number

    minColumnNum

    Sequence of load checks for minimum column number

    columnsExist

    Sequence of load checks to verify columns existence

    schemaMatch

    Sequence of load checks to verify if source schema matches target one

  5. final case class MinColNumLoadCheckConfig(id: ID, description: Option[NonEmptyString], source: NonEmptyString, option: PositiveInt, isCritical: Boolean = false, metadata: Seq[SparkParam] = Seq.empty) extends LoadCheckConfig with Product with Serializable

    Minimum column number load check configuration

    Minimum column number load check configuration

    id

    Load check ID

    description

    Load check description

    source

    Source ID to be checked

    option

    Minimum number of columns

    isCritical

    Flag if check is critical

    metadata

    List of metadata parameters specific to this load check

  6. final case class SchemaMatchLoadCheckConfig(id: ID, description: Option[NonEmptyString], source: NonEmptyString, schema: NonEmptyString, ignoreOrder: Boolean = false, isCritical: Boolean = false, metadata: Seq[SparkParam] = Seq.empty) extends LoadCheckConfig with Product with Serializable

    Schema match load check configuration

    Schema match load check configuration

    id

    Load check ID

    description

    Load check description

    source

    Source ID to be checked

    schema

    Schema ID to match with

    ignoreOrder

    If true than order of columns in schemas is ignored

    isCritical

    Flag if check is critical

    metadata

    List of metadata parameters specific to this load check

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  16. def toString(): String
    Definition Classes
    AnyRef → Any
  17. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  18. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  19. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped