2 years ago

#66090

test-img

aschmied

Google DataFlow pipeline stuck draining with large negative data freshness for beam.io.WriteToBigQuery

I have a DataFlow pipeline that occasionally (I've seen it twice now) gets stuck draining. It has been draining for 4 days now. Execution Details show a large negative output data freshness for a beam.io.WriteToBigQuery sink:

enter image description here

I'm not certain that the negative data freshness is causing the hang, but it feels like a good place to start. Looking for suggestions for how to diagnose. I have not found anything by Googling.

google-cloud-dataflow

0 Answers

Your Answer

Accepted video resources