2 years ago
#42319
HyeonPhil Youn
Tensorflow tf.data.Dataset interleave() returns Unexpected result
I was following some another example from this link which is about the tf.data.Dataset.interleave()
method.
import tensorflow as tf
import tensorflow.keras as keras
def do_range(i):
for j in tf.range(i):
yield j
ds = tf.data.Dataset.range(10).interleave(
lambda ind: tf.data.Dataset.from_generator(do_range, args=(ind,), output_types=tf.int64))
I tried to understand this example, so I started with the range(3)
, range(4)
, ..., and so on.
For example,
ds = tf.data.Dataset.range(3).interleave(
lambda ind: tf.data.Dataset.from_generator(do_range, args=(ind,), output_types=tf.int64))
res = [x for x in ds.as_numpy_iterator()]
# This returns [0, 0, 1]
ds = tf.data.Dataset.range(4).interleave(
lambda ind: tf.data.Dataset.from_generator(do_range, args=(ind,), output_types=tf.int64))
res = [x for x in ds.as_numpy_iterator()]
# This returns [0, 0, 0, 1, 1, 2]
ds = tf.data.Dataset.range(5).interleave(
lambda ind: tf.data.Dataset.from_generator(do_range, args=(ind,), output_types=tf.int64))
res = [x for x in ds.as_numpy_iterator()]
# This returns [0, 0, 0, 0, 1, 1, 1, 2, 2, 3]
Until range(9)
, it returns the dataset I expected.
But from range(10)
, it doesn't return the dataset I expected.
ds = tf.data.Dataset.range(10).interleave(
lambda ind: tf.data.Dataset.from_generator(do_range, args=(ind,), output_types=tf.int64))
res = [x for x in ds.as_numpy_iterator()]
# This returns [0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 2, 2, 2, 2, 2, 2, 1, 3, 3, 3, 3, 3, 2, 4, 4, 4, 4, 3, 5, 5, 5, 4, 6, 6, 5, 7, 6, 7, 8]
# Notice that 0s, 1s, 2s are not gathered..
What is going on here? Isn't this should be [0, ..., 0, 1, ..., 1, 2, ..., 2, ..., 8]
?
python
tensorflow
tensorflow-datasets
interleave
0 Answers
Your Answer