Type Members

1.  type AccuracyDouble = Refined[Double, OpenClosed[Double(0.0), Double(1.0)]]
2.  case class DateFormat(pattern: String) extends Product with Serializable

   DateFormat class is used to store both date-time pattern and corresponding formatter.

   DateFormat class is used to store both date-time pattern and corresponding formatter. Such construction allows to verify if pattern is convertable to formatter at the point of configuration reading and also supports backwards conversion (when writing configuration). The issue with backwards conversion is due to DateTimeFormatter does not store string representation of original datetime pattern.

   pattern

   - datetime pattern string

   Note

   DataTimeFormatter is not serializable, therefore, it is marked as @transient to avoid serialization errors in Spark Tasks.

3.  type DoubleElemStringSeq = Refined[Seq[String], Size[Equal[Int(2)]]]
4.  case class Email(value: String) extends Product with Serializable

   Email class is used to wrap email addresses and provide email validation during configuration parsing.

   Email class is used to wrap email addresses and provide email validation during configuration parsing.

   value

   Validated email address

5.  type EncryptionKey = Refined[String, MatchesRegex[String("^.{32}.*$")]]

   Refinement for string holding secret key for sensitive data encryption.

   Refinement for string holding secret key for sensitive data encryption. This string should contain at least 32 characters.

6.  type FixedShortColumn = Refined[String, MatchesRegex[String("^[^\\n\\r\\t:]+:\\d+$")]]
7.  case class ID(value: String) extends Product with Serializable

   All IDs are parsers via SparkSqlParser so they are valid SparkSQL identifiers This is required to eliminate any issues with interoperability with Spark.

   All IDs are parsers via SparkSqlParser so they are valid SparkSQL identifiers This is required to eliminate any issues with interoperability with Spark. For example, source IDs can be used to create virtual source: in this case parent source are registered as temporary views in Spark.

   value

   Validated ID value

8.  type MMRecipient = Refined[String, MatchesRegex[String("^(@|#)[a-zA-Z][a-zA-Z0-9.\\-_]+$")]]
9.  type MultiElemStringSeq = Refined[Seq[String], MinSize[Int(2)]]
10.  type NonEmptyDoubleSeq = Refined[Seq[Double], NonEmpty]

    Refinements for numeric sequences:

11.  type NonEmptyStringSeq = Refined[Seq[String], NonEmpty]

    Various refinements for string sequences:

12.  type NonEmptyURISeq = Refined[Seq[URI], NonEmpty]
13.  type PercentileDouble = Refined[Double, Closed[Double(0.0), Double(1.0)]]
14.  type Port = Refined[Int, Closed[Int(0), Int(9999)]]
15.  type PositiveInt = Refined[Int, Positive]
16.  type RegexPattern = Refined[String, Regex]
17.  type SingleElemStringSeq = Refined[Seq[String], Size[Equal[Int(1)]]]
18.  type SparkParam = Refined[String, MatchesRegex[String("^\\S+?\\=[\\S\\s]+$")]]
19.  type ThreeElemIntSeq = Refined[Seq[Int], Size[Equal[Int(3)]]]
20.  type URI = Refined[String, Uri]
21.  type URL = Refined[String, Url]