相关文章推荐
幸福的墨镜  ·  Linux下解决./configure ...·  1 年前    · 
英姿勃勃的烤面包  ·  R Error in x$ed : $ ...·  1 年前    · 

在CNN自动编码器中用预训练vgg19 tensorflow,Keras定义自定义损失(感知损失)。

0 人关注

我想在keras中建立的自动编码器中定义感知损失。 我的自动编码器是这样的。

Encoder:

input_encoder = Input((32,32,3),name = 'encoder_input')
encoder = Conv2D(16,(3,3),activation = 'relu',name = 'encoder_layer1')(input_encoder)
encoder = Flatten(name = 'encoder_layer2')(encoder)
latent_encoding = Dense(128 , activation = 'relu', name = 'encoder_layer3')(encoder)
Encoder = Model(inputs= [input_encoder], outputs=[latent_encoding],name = 'Encoder')

Decoder :

input_decoder = Input(128,name = 'decoder_input')
decoder = Reshape((1, 1, 128),name = 'decoder_layer1')(input_decoder)
decoder = Conv2DTranspose(64, (2,2), activation='relu' , name = 'decoder_layer2')(decoder)
decoder = UpSampling2D(8 ,name = 'decoder_layer3')(decoder)
decoder = Conv2DTranspose(3, (9,9), activation='relu' , name = 'decoder_layer2')(decoder)
Decoder = Model(inputs= [input_decoder], outputs=[decoder ],name = 'Decoder')

自动编码器:

input = Input((32,32,3),name = 'input')
latent = Encoder(input)
output = Decoder(latent)
AE = Model(inputs= [input], outputs=[output ],name = 'AE')

现在我定义了新的损失函数perceptual_loss,预训练vgg19,就像这样,我得到输入图像和重建图像来预训练vgg19,从vgg19的某些层得到结果,然后我用两个向量的减法作为vgg19中该层的误差,然后我用各层的误差加权和来计算总误差。

selected_layers = ['block1_conv1', 'block2_conv2',"block3_conv3" ,'block4_conv3','block5_conv4']
selected_layer_weights = [1.0, 4.0 , 4.0 , 8.0 , 16.0]
def perceptual_loss(input_image , reconstruct_image):
    vgg = VGG19(weights='imagenet', include_top=False, input_shape=(32,32,3))
    vgg.trainable = False
    outputs = [vgg.get_layer(l).output for l in selected_layers]
    model = Model(vgg.input, outputs)
    h1_list = model(input_image)
    h2_list = model(reconstruct_image)
    rc_loss = 0.0
    for h1, h2, weight in zip(h1_list, h2_list, selected_layer_weights):
        h1 = K.batch_flatten(h1)
        h2 = K.batch_flatten(h2)
        rc_loss = rc_loss + weight * K.sum(K.square(h1 - h2), axis=-1)
    return rc_loss

然后我编译AE。

rmsprop = RMSprop(learning_rate=0.00025)
AE.compile(loss= perceptual_loss, optimizer= rmsprop)

但当我想适合AE时。

history = AE.fit(train_images, train_images,
                          epochs= 2,
                          verbose=1)

i get error

ValueError: tf.function-decorated function tried to create variables on non-first call.

请帮助我,谢谢

update :

我根据@Mr. For Example的回答更新了损失函数,但我得到了新的错误。 现在我有了损失函数。

define perceptual_loss selected_layers = ['block1_conv1', 'block2_conv2',"block3_conv3" ,'block4_conv3','block5_conv4'] selected_layer_weights = [1.0, 4.0 , 4.0 , 8.0 , 16.0] vgg = VGG19(weights='imagenet', include_top=False, input_shape=(32,32,3)) vgg.trainable = False outputs = [vgg.get_layer(l).output for l in selected_layers] model = Model(vgg.input, outputs) def perceptual_loss(input_image , reconstruct_image): h1_list = model(input_image) h2_list = model(reconstruct_image) rc_loss = 0.0 for h1, h2, weight in zip(h1_list, h2_list, selected_layer_weights): h1 = K.batch_flatten(h1) h2 = K.batch_flatten(h2) rc_loss = rc_loss + weight * K.sum(K.square(h1 - h2), axis=-1) return rc_loss

i get new error :

ValueError                                Traceback (most recent call last)
<ipython-input-16-3133696ab8be> in <module>()
----> 1 VAE.fit(train_images[:5],train_images[:5],epochs=2,verbose=1)
2 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
   1098                 _r=1):
   1099               callbacks.on_train_batch_begin(step)
-> 1100               tmp_logs = self.train_function(iterator)
   1101               if data_handler.should_sync:
   1102                 context.async_wait()
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
    826     tracing_count = self.experimental_get_tracing_count()
    827     with trace.Trace(self._name) as tm:
--> 828       result = self._call(*args, **kwds)
    829       compiler = "xla" if self._experimental_compile else "nonXla"
    830       new_tracing_count = self.experimental_get_tracing_count()
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
    862       results = self._stateful_fn(*args, **kwds)
    863       if self._created_variables:
--> 864         raise ValueError("Creating variables on a non-first call to a function"
    865                          " decorated with tf.function.")
    866       return results
ValueError: Creating variables on a non-first call to a function decorated with tf.function.

update 2

正如@Navid所说,我在损失函数前加上@tf.function,错误就消失了。

define perceptual_loss selected_layers = ['block1_conv1', 'block2_conv2',"block3_conv3" ,'block4_conv3','block5_conv4'] selected_layer_weights = [1.0, 4.0 , 4.0 , 8.0 , 16.0] vgg = VGG19(weights='imagenet', include_top=False, input_shape=(32,32,3)) vgg.trainable = False outputs = [vgg.get_layer(l).output for l in selected_layers] model = Model(vgg.input, outputs) @tf.function def perceptual_loss(input_image , reconstruct_image): h1_list = model(input_image) h2_list = model(reconstruct_image) rc_loss = 0.0 for h1, h2, weight in zip(h1_list, h2_list, selected_layer_weights): h1 = K.batch_flatten(h1) h2 = K.batch_flatten(h2) rc_loss = rc_loss + weight * K.sum(K.square(h1 - h2), axis=-1) return rc_loss ```
python
tensorflow
keras
loss-function
autoencoder
alish
alish
发布于 2020-12-29
2 个回答
Navid
Navid
发布于 2020-12-29
已采纳
0 人赞同

只要在损失函数之外创建模型,并在损失函数的定义之前使用 @tf.function

Mr. For Example
Mr. For Example
发布于 2020-12-29
0 人赞同

你不应该在损失函数中创建模型,而应该做类似的事情。

selected_layers = ['block1_conv1', 'block2_conv2',"block3_conv3" ,'block4_conv3','block5_conv4']
selected_layer_weights = [1.0, 4.0 , 4.0 , 8.0 , 16.0]
vgg = VGG19(weights='imagenet', include_top=False, input_shape=(32,32,3))
vgg.trainable = False
outputs = [vgg.get_layer(l).output for l in selected_layers]
model = Model(vgg.input, outputs)
@tf.function
def perceptual_loss(input_image , reconstruct_image):
    h1_list = model(input_image)
    h2_list = model(reconstruct_image)
    rc_loss = 0.0
    for h1, h2, weight in zip(h1_list, h2_list, selected_layer_weights):
        h1 = K.batch_flatten(h1)