implicit object ErrorCollHiveTargetProcessor extends TargetProcessor[ErrorCollHiveTargetConfig, DataFrame] with ErrorDataFrameBuilder[ErrorCollHiveTargetConfig] with HiveWriter[ErrorCollHiveTargetConfig] with MetricErrorsTargetType
- Alphabetic
- By Inheritance
- ErrorCollHiveTargetProcessor
- MetricErrorsTargetType
- HiveWriter
- OutputWriter
- ErrorDataFrameBuilder
- BuildHelpers
- TargetBuilder
- TargetProcessor
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
build(target: ErrorCollHiveTargetConfig, results: ResultSet)(implicit settings: AppSettings, spark: SparkSession): Result[DataFrame]
Build target output given the target configuration
Build target output given the target configuration
- target
Target configuration
- results
All job results
- settings
Implicit application settings object
- spark
Implicit spark session object
- returns
Target result in form of Spark DataFrame
- Definition Classes
- ErrorDataFrameBuilder → TargetBuilder
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
filterErrors(errors: Seq[ResultMetricError], requested: Seq[String], dumpSize: Int): Seq[ResultMetricError]
Filters errors to be send in target.
Filters errors to be send in target.
- errors
Sequence of all collected metric errors
- requested
Sequence of metric IDs for which metric errors are requested
- dumpSize
Maximum number of errors to be collected per each metric
- returns
Filtered sequence of metric errors
- Definition Classes
- BuildHelpers
- Note
If requested metric IDs are not provided (sequence is empty) then errors for all metrics are returned.
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
def
process(target: ErrorCollHiveTargetConfig, results: ResultSet)(implicit jobId: String, settings: AppSettings, spark: SparkSession, connections: Map[String, DQConnection]): Result[String]
Safely process target given the target configuration.
Safely process target given the target configuration.
- target
Target configuration
- results
All job results
- returns
"Success" string in case of successful operation or a list of errors.
- Definition Classes
- TargetProcessor
-
def
readTemplate(uri: String): Option[String]
Safely reads template from file
Safely reads template from file
- uri
Path to a template file
- returns
Template string read from path or None if read operation wasn't successful.
- Definition Classes
- BuildHelpers
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
val
targetType: String
- Definition Classes
- MetricErrorsTargetType
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
def
write(target: ErrorCollHiveTargetConfig, result: DataFrame)(implicit jobId: String, settings: AppSettings, spark: SparkSession, connections: Map[String, DQConnection]): Result[String]
Writes result to required output channel given the output configuration.
Writes result to required output channel given the output configuration.
- target
Output configuration
- result
Result to be written
- returns
"Success" string in case of successful write operation or a list of errors.
- Definition Classes
- HiveWriter → OutputWriter