Packages

o

org.checkita.dqf.readers

SchemaReaders

object SchemaReaders

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SchemaReaders
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. sealed trait SchemaReader[T <: SchemaConfig] extends AnyRef

    Base schema reader trait

    Base schema reader trait

    T

    Type of schema configuration

  2. implicit class SchemaReaderOps[T <: SchemaConfig] extends AnyRef

    Implicit conversion for schema configurations that enables read method for them.

    Implicit conversion for schema configurations that enables read method for them.

    T

    Type of schema configuration

  3. final case class SourceSchema(id: String, schema: StructType, columnWidths: Seq[Int] = Seq.empty, rawAvro: Option[String] = None) extends Product with Serializable

    Source schema: standard representation for all explicitly defined schemas: all schemas are converted to spark StructType.

    Source schema: standard representation for all explicitly defined schemas: all schemas are converted to spark StructType.

    id

    Schema ID

    schema

    Schema in form of StructType

    columnWidths

    Sequence of column widths in order (to support fixed-width file reading)

    rawAvro

    Raw avro schema representation (string in JSON format). In case when Avro schema is read from file, the original schema is stored to prevent unnecessary conversions.

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  16. def toString(): String
    Definition Classes
    AnyRef → Any
  17. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  18. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  19. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  20. implicit object AnySchemaReader extends SchemaReader[SchemaConfig]

    General schema reader: invokes read method from schema reader that matches provided schema configuration

  21. implicit object AvroSchemaReader extends SchemaReader[AvroSchemaConfig]

    Avro schema reader: reads schema from .avsc file

  22. implicit object DelimitedSchemaReader extends SchemaReader[DelimitedSchemaConfig]

    Delimited schema reader (schema defined explicitly in job configuration file)

  23. implicit object FixedFullSchemaReader extends SchemaReader[FixedFullSchemaConfig]

    Reader for fixed schema with fully defined columns (name, type, width)

  24. implicit object FixedShortSchemaReader extends SchemaReader[FixedShortSchemaConfig]

    Reader for fixed schema with shortly defined columns (name, width)

    Reader for fixed schema with shortly defined columns (name, width)

    Note

    Type of columns is always a StringType

  25. implicit object HiveSchemaReader extends SchemaReader[HiveSchemaConfig]

    Hive schema reader: retrieves schema for given Hive schema.table

  26. implicit object RegistrySchemaReader extends SchemaReader[RegistrySchemaConfig]
  27. object SourceSchema extends Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped