tf读取数据

为啥数据用tf存储呢？存储读取都麻烦，真是垃圾，不知道这些人咋想的，只有一个解释：装*。

老子不怕这玩意。从没有见过这个垃圾的玩意，定义的关键字参数与使用的不同，而且是错位用了，你这样还默认参数有个屁用【写这种代码的直接可以去*了】

指定关键字了不用同样的名字参数，至少应该差不多吧，比如参数为a,b=2,c=3，你咋能使用b,c，你特么是不是有*？？

def func(a,b=2,c=3):
...
    return
b=...
c=...
func(b,c)

【恕我直言，tf读数据就是傻*，pandas一下解决了。除非你的数据只有tf能读】

对于每一行为一条数据的东西，tf有专门的解读类，filenames可以为一个或者多个文件的列表

data=tf.data.TextLineDataset(filenames)
data.map(lambda:line,linedecode(line))

其中linedecode是自己定义的函数，用于解读每行的数据

如果每行字符串需要decode，需要知道分隔符sep才能string_slit，然后再进行string_to_number即可

其中的细节在于：rank为1，如果rank为0，增加[]即可解决，否则出现如下错误：

Traceback (most recent call last):
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1607, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be rank 1 but is rank 0 for 'StringSplit_3/StringSplit' (op: 'StringSplit') with input shapes: [], [].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<pyshell#28>", line 1, in <module>
    tf.string_split(next_ele.values[1],',')
  File "D:\python\lib\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "D:\python\lib\site-packages\tensorflow_core\python\ops\ragged\ragged_string_ops.py", line 570, in string_split
    source, sep=sep, skip_empty=skip_empty, delimiter=delimiter)
  File "D:\python\lib\site-packages\tensorflow_core\python\ops\string_ops.py", line 245, in string_split
    source, delimiter=delimiter, skip_empty=skip_empty)
  File "D:\python\lib\site-packages\tensorflow_core\python\ops\gen_string_ops.py", line 1357, in string_split
    skip_empty=skip_empty, name=name)
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\op_def_library.py", line 794, in _apply_op_helper
    op_def=op_def)
  File "D:\python\lib\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3357, in create_op
    attrs, op_def, compute_device)
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3426, in _create_op_internal
    op_def=op_def)
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1770, in __init__
    control_input_ops)
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1610, in _create_c_op
    raise ValueError(str(e))
ValueError: Shape must be rank 1 but is rank 0 for 'StringSplit_3/StringSplit' (op: 'StringSplit') with input shapes: [], [].
>>> tf.string_to_number(tf.string_split(next_ele.values[1],','))
Traceback (most recent call last):
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1607, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be rank 1 but is rank 0 for 'StringSplit_4/StringSplit' (op: 'StringSplit') with input shapes: [], [].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<pyshell#29>", line 1, in <module>
    tf.string_to_number(tf.string_split(next_ele.values[1],','))
  File "D:\python\lib\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "D:\python\lib\site-packages\tensorflow_core\python\ops\ragged\ragged_string_ops.py", line 570, in string_split
    source, sep=sep, skip_empty=skip_empty, delimiter=delimiter)
  File "D:\python\lib\site-packages\tensorflow_core\python\ops\string_ops.py", line 245, in string_split
    source, delimiter=delimiter, skip_empty=skip_empty)
  File "D:\python\lib\site-packages\tensorflow_core\python\ops\gen_string_ops.py", line 1357, in string_split
    skip_empty=skip_empty, name=name)
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\op_def_library.py", line 794, in _apply_op_helper
    op_def=op_def)
  File "D:\python\lib\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3357, in create_op
    attrs, op_def, compute_device)
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3426, in _create_op_internal
    op_def=op_def)
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1770, in __init__
    control_input_ops)
  File "D:\python\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1610, in _create_c_op
    raise ValueError(str(e))
ValueError: Shape must be rank 1 but is rank 0 for 'StringSplit_4/StringSplit' (op: 'StringSplit') with input shapes: [], [].

经过string_split的数据一般有下面的数据结构：我们通常只需values即可，而且values是列表

    tf.SparseTensor(indices=[[0, 0], [0, 1], [1, 0], [1, 1], [1, 2]],
                    values=['hello', 'world', 'a', 'b', 'c']
                    dense_shape=[2, 3])

dense_shape表示数据个数，2表示第一个字符串分割为多少个字符/串，3表示第二个字符串分割为多少个字符/串

此外还可以对此数据生成迭代器，每次取batch_size个，也可指定epochs，看起来似乎也挺方便的。

 

另外有相关问题可以加入一个QQ群讨论，不设微信群

语音图像视频深度-学习QQ群：868373192 

视频推荐深度-学习QQ群：277356808