源代码如下:
(这是代码内调用模型设置的路径,大家根据自己代码的情况做出相应的修改)
# bert配置
config_path = './publish/bert_config.json'
checkpoint_path = './publish/model.ckpt'
dict_path = './publish/vocab.txt'
然后报错:
Could not open .\publish\model.ckpt: Data loss: not an sstable (bad magic number):perhaps your file is in a different file format and you need to use a different restore operator?
解决方案:
代码:
# bert配置
config_path = './publish/bert_config.json'
checkpoint_path =tf.train.latest_checkpoint('./publish/')
#checkpoint_path = './publish/model.ckpt'
dict_path = './publish/vocab.txt'
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file /home/wpq/data/model.ckpt.data-00000-of-00001:
Data loss: not an sstable (bad magic number): perhaps your file is ...
预训练过程中主要修改了两个文件,一个是create_pretraining_
data和run_pretrain.
前者运行需要读取一个txt文件,记得改成utf-8。在前者代码里面,将读进来的txt转化成了tfrecod形式,存进了example.tfrecord,在后者的程序里会调用它,来预训练。
两个代码程序都需要保证读取成功,如果路径没问题,仍然
报错,
可以去路径下看example.tfrecord有没有正确生成。
tensorf
low.
python.framework.e
rrors_impl.
DataLossE
rror: corrupted record at XXXX
DataLossE
rror (see above for traceback): corrupted record at XXXXXXX
错误的可能原因:
tf record文件损坏,可以重新下载或者生成record文件;