Packages

o

org.checkita.dqf.readers

SourceReaders

object SourceReaders

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SourceReaders
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. sealed trait SimpleFileReader extends AnyRef
  2. sealed trait SourceReader[T <: SourceConfig] extends AnyRef

    Base source reader trait

    Base source reader trait

    T

    Type of source configuration

  3. implicit class SourceReaderOps[T <: SourceConfig] extends AnyRef

    Implicit conversion for source configurations to enable read and readStream methods for them.

    Implicit conversion for source configurations to enable read and readStream methods for them.

    T

    Type of source configuration

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  16. def toString(): String
    Definition Classes
    AnyRef → Any
  17. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  18. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  19. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  20. implicit object AnySourceReader extends SourceReader[SourceConfig]

    Generic regular source reader that calls specific reader depending on the source configuration type.

  21. implicit object AvroFileSourceReader extends SourceReader[AvroFileSourceConfig] with SimpleFileReader

    Avro file source reader: reads avro file with optional explicit schema.

    Avro file source reader: reads avro file with optional explicit schema.

    Note

    In order to read avro file source it is required to provide map of source schemas.

  22. implicit object CustomSourceReader extends SourceReader[CustomSource]
  23. implicit object DelimitedFileSourceReader extends SourceReader[DelimitedFileSourceConfig]

    Delimited file source reader: reads delimited (csv, tsv, etc.) file.

    Delimited file source reader: reads delimited (csv, tsv, etc.) file. Schema may be inferred from file header or provided explicitly in job configuration file and referenced in this source by its ID.

    Note

    In order to read delimited file source it is required to provide map of source schemas

  24. implicit object FixedFileSourceReader extends SourceReader[FixedFileSourceConfig]

    Fixed file source reader: reads fixed-width file provided with schema ID corresponding to this file contents.

    Fixed file source reader: reads fixed-width file provided with schema ID corresponding to this file contents.

    Note

    In order to read fixed file source it is required to provide map of source schemas.

  25. implicit object GreenplumSourceReader extends SourceReader[GreenplumSourceConfig]

    Greenplum source reader: reads source from pivotal Connection

    Greenplum source reader: reads source from pivotal Connection

    Note

    In order to read greenplum source it is required to provided map of valid connections

  26. implicit object HiveSourceReader extends SourceReader[HiveSourceConfig]

    Hive source reader: reads table from Hive.

    Hive source reader: reads table from Hive. Can also read only required partitions from table given the list partition columns and their values to read.

  27. implicit object KafkaSourceReader extends SourceReader[KafkaSourceConfig]

    Kafka source reader: reads topic from Kafka broker

    Kafka source reader: reads topic from Kafka broker

    Note

    In order to read kafka source it is required to provided map of valid connections

  28. implicit object OrcFileSourceReader extends SourceReader[OrcFileSourceConfig] with SimpleFileReader

    Orc file source reader: reads orc files.

  29. implicit object ParquetFileSourceReader extends SourceReader[ParquetFileSourceConfig] with SimpleFileReader

    Parquet file source reader: reads parquet files.

  30. implicit object TableSourceReader extends SourceReader[TableSourceConfig]

    Table source reader: reads source from JDBC Connection (Postgres, Oracle, etc)

    Table source reader: reads source from JDBC Connection (Postgres, Oracle, etc)

    Note

    In order to read table source it is required to provided map of valid connections

Inherited from AnyRef

Inherited from Any

Ungrouped