tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file /home/wpq/data/model.ckpt.data-00000-of-00001:
Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you
need to use a different restore operator?
解决方案：
I have fixed the issue by this:
replace
model.ckpt
the
model.ckpt.
data-00000-of-00001
即使用
model.ckpt替换
model.ckpt.
data-00000-of-00001，去掉
model.ckpt.
data-00000-of-00001的后缀即可
参考：
https://github.com/tensorflow/models/issues/2675
报错：tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file /home/wpq/data/model.ckpt.data-00000-of-00001:Data loss: not an sstable (bad magic number): perhaps your file is ...
Table
Builder builder = new
Table
Builder();
builder.setUseBloomFilter(true);
builder.put("abc".getBytes(), "123".getBytes());
// Put some more
data
builder.writeTo(new
File
("
data
.
ss
t"));
使用键值存储：
Table
table
=
Table
Reader.getReader().from(new
File
("
data
.
ss
t"));
byte[] value =
table
.get("abc".getBytes());
$ git clone git@github.com:instaclustr/ca
ss
andra-
ss
table
-tools.git
$ cd ca
ss
andra-
ss
table
-tools
# Select the correct branch for major version (default is ca
ss
andra-4.0)
$ git checkout ca
ss
andra-4.0
$ mvn clean install
将ic-
ss
table
-tools.jar复制到Ca
ss
andra JAR文件夹，例如。 /usr/share/ca
ss
andra/lib
将bin/ic-
ss
table
-tools脚本复制到$PATH
我们还提供RPM和DEB软件包。 如果安装它们，显然不需要执行上述步骤。
$ ./b
import tensorflow as tf
from tensorflow.python import pywrap_tensorflow
checkpoint_path='model.ckpt-5000'#your ckpt path
reader=pywrap_tensorflow.NewCheckpointReader(checkpoint_path)
var_to_shape_map=reader.get_variable_to_shape_map()
alexnet={}
alexnet_layer
# bert配置
config_path = './publish/bert_config.json'
checkpoint_path = './publish/model.ckpt'
dict_path = './publish/vocab.txt'
然后报错：
Could not open .\publish\model.ckpt:
Data
loss
: not an
ss
table
(
bad
magic
number
):perha
Data
loss
: not an
ss
table
(
bad
magic
number
): perhaps your
file
is in a
diff
erent
file
format and you
need to use a
diff
erent restore operator?
绝对偏移量是64位字（ Word64 ），而相对偏移量是32位（ Word32 ）。 所有值都按big-endian字节顺序序列化。
标头包含文件中的第一个数据，还包含
SS
Table
上的元数据以及块索引的偏移量。
VERSION :: Word32
NUM-BLOCKS :: Word32
INDEX-OFFSET :: Word64
KEY-LENGTH :: Word32
ENTRY-LENGTH :: Word32
KEY-BYTES :: [Word8]
ENTRY-BYTES :: [Word8]
块索引输入
KEY-LENGTH :: Word32
BLOCK-
预训练过程中主要修改了两个文件，一个是create_pretraining_
data
和run_pretrain.
前者运行需要读取一个txt文件，记得改成utf-8。在前者代码里面，将读进来的txt转化成了tfrecod形式，存进了example.tfrecord,在后者的程序里会调用它，来预训练。
两个代码程序都需要保证读取成功，如果路径没问题，仍然报错，
可以去路径下看example.tfrecord有没有正确生成。
