1 year ago

#76925

test-img

Rosa Xue

You may get a different result due to the upgrading of Spark 3.0: Fail to parse 'JUN 2001' in the new parser

I'm working on converting a string to date in SparkSQL. The string format is "Jun 2001', I tried to convert to date format as 'MMMyyyy' by using the query below

SELECT 
       TO_DATE(CAST('JUN 2001'AS STRING),'MMMyyyy')

However, it kept giving me error as below,

*Error in SQL statement: SparkUpgradeException: You may get a different result due to the upgrading of Spark 3.0: Fail to parse 'JUN 2001' in the new parser. You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0, or set to CORRECTED and treat it as an invalid datetime string.*

Is there any solution for it?

Thanks!

sql

apache-spark-sql

azure-databricks

0 Answers

Your Answer

Accepted video resources