1 year ago
#70072

Борис Маринов
Is it possible to load constrains from file (csv, txt) to Deequ Checks?
Is it possible to save suggested constrains to file and then load them as cheks? I was able to do it without saving them with next code
val allConstraints = suggestionResult.constraintSuggestions.flatMap {
case (_, suggestions) =>
suggestions.map {
_.constraint
}
}.toSeq
val generatedCheck = Check(CheckLevel.Error, "generated constraints", allConstraints)
val verificationResult: VerificationResult = {
VerificationSuite()
.onData(tested_df)
.addCheck(generatedCheck)
.run()
}
However I want to save them to file and apply them later when needed? Is there any way to do this?
dataframe
scala
apache-spark
amazon-deequ
0 Answers
Your Answer