1 year ago

#70072

test-img

Борис Маринов

Is it possible to load constrains from file (csv, txt) to Deequ Checks?

Is it possible to save suggested constrains to file and then load them as cheks? I was able to do it without saving them with next code

val allConstraints = suggestionResult.constraintSuggestions.flatMap {
      case (_, suggestions) =>
        suggestions.map {
          _.constraint
        }
    }.toSeq

val generatedCheck = Check(CheckLevel.Error, "generated constraints", allConstraints)

val verificationResult: VerificationResult = {
        VerificationSuite()
          .onData(tested_df)
          .addCheck(generatedCheck)
          .run()
      }

However I want to save them to file and apply them later when needed? Is there any way to do this?

dataframe

scala

apache-spark

amazon-deequ

0 Answers

Your Answer

Accepted video resources