python (12.9k questions)

javascript (9.2k questions)

reactjs (4.7k questions)

java (4.2k questions)

java (4.2k questions)

c# (3.5k questions)

c# (3.5k questions)

html (3.3k questions)

Questions - iceberg

Apache Iceberg table format to ADLS / azure data lake

I am trying to find some integration to use iceberg table format on adls /azure data lake to perform crud operations. Is it possible to not use any other computation engine like spark to use it on azu...
test-img

John

amazon-s3

azure-data-lake

azure-data-lake-gen2

trino

iceberg

Votes: 0

Answers: 2

Latest Answer

spark can use Iceberg with the abfs connector, hdfs, even local files. you just need the classpath and authentication right
test-img

stevel

Iceberg: How to quickly traverse a very large table

I'm new to iceberg, and i have a question about query big table. We have a Hive table with a total of 3.6 million records and 120 fields per record. and we want to transfer all the records in this tab...
test-img

xujin

apache-spark

hive

iceberg

Votes: 0

Answers: 1

Latest Answer

In addition to reading row by row, here is another idea. If your target database can import files directly, try retrieving files from Iceberg and importing them directly to the database. Example code ...
test-img

liliwei

Writing multiple partition specs to Apache Iceberg table

I would like to write an Iceberg table with a different partition spec than the default table settings so that when I run data compaction the data would be compacted according to the default spec (as ...
test-img

Shimon Steinitz

apache-spark

apache-spark-sql

iceberg

Votes: 0

Answers: 1

Latest Answer

According to the docs Streaming writes, I think you can't specify partitions while writing stream. Maybe you can try adding the new partition field before writing, using ALTER TABLE SQL extensions ava...
test-img

blackbishop

Posts

Questions

Blogs

Jobs