python (12.9k questions)

javascript (9.2k questions)

reactjs (4.7k questions)

java (4.2k questions)

java (4.2k questions)

c# (3.5k questions)

c# (3.5k questions)

html (3.3k questions)

Questions - rocksdb

Databricks Autoloader is getting stuck and does not pass to the next batch

I have a simple job scheduled every 5 min. Basically it listens to cloudfiles on storage account and writes them into delta table, extremely simple. The code is something like this: df = (spark .rea...
test-img

Maksym

spark-streaming

databricks

autoload

rocksdb

mini-batch

Votes: 0

Answers: 1

Latest Answer

I resolved it by using .option('cloudFiles.useIncrementalListing', 'false') My filenames are composed of flowname + timestamp, like this: flow_name_2022-01-18T14-19-50.018Z.json So my guess is: some c...
test-img

Maksym

Deleting RocksDB Test-Ledger files in Solana Anchor ReactJS projects

I am facing a re-occurring problem specifically with Solana Anchor ReactJS projects where I can't commit my projects into Github because my git keeps pulling in my test-ledger rocksdb files. Why is Ro...
test-img

Mabel Oza

git

rocksdb

solana

anchor-solana

Votes: 0

Answers: 0

How to use cloud storage (blob stores) with Rocksdb

My use case: A single-node out-of-memory "big dict" (or "big map"). The total size is too large for memory, e.g. 20gb, but is ok for single-node disk. Due to the total size, it's u...
test-img

zpz

caching

blobstore

rocksdb

Votes: 0

Answers: 2

Latest Answer

RocksDB allows you to define your own FileSystem or Env, which you can implement the interaction layer with whatever special filesystem you want. So it's possible, but you need implement or define the...
test-img

Jay Zhuang

Posts

Questions

Blogs

Jobs