python (12.9k questions)

javascript (9.2k questions)

reactjs (4.7k questions)

java (4.2k questions)

java (4.2k questions)

c# (3.5k questions)

c# (3.5k questions)

html (3.3k questions)

Questions - wal

Spark structured streaming job failing after running for 3-4 days with java.io.FileNotFoundException: File does not exist

We have a spark structured streaming job written in scala running in production which reads from a kafka topic and writes to HDFS sink. Triggertime is 60 Seconds. The job has been deployed 4months bac...
test-img

sam.ban

apache-spark

hadoop-yarn

spark-structured-streaming

wal

Votes: 0

Answers: 0

Time to remove WAL from Postrgres

Just curious as to why it takes so long to remove WAL from a table in Postgres ALTER TABLE tablename SET UNLOGGED Took more than an hour on a very large table but with very little CPU utilization. Is...
test-img

TnTech

postgresql

wal

Votes: 0

Answers: 1

Latest Answer

https://wiki.postgresql.org/wiki/Improve_the_performance_of_ALTER_TABLE_SET_LOGGED_UNLOGGED_statement "However this design lead us to performance problems with large relations because we need to ...
test-img

Ron Warshawsky

Debezium `source.lsn` field value in snapshot records(when `__op=r`)

I can't find an information in Debezium Postgresql connector config documentation page regarding the source.lsn field value for snapshot events(events having __op=r). I've ran a test on my dev environ...
test-img

Suren Aznauryan

postgresql

apache-kafka-connect

debezium

wal

Votes: 0

Answers: 1

Latest Answer

How does debezium get a non-null lsn The source.lsn value for a new snapshot is sourced from the consistent_point that available via the newly created replication slot when the connector began. Thi...
test-img

Naros

Posts

Questions

Blogs

Jobs