JPA系列(三):Spring Jpa save问题缓存总结
参考博客:
最近自己因为不了解Spring jpa save的缓存机制而导致的问题,故在此记录如下。
一、问题:
在代码开发中,会将接口接收的参数信息先存储在第三方服务(服务B中),然后根据B服务存储返回结果中的id(此id由B服务内部生成),对参数信息实体设置该id属性。然后再将对应信息存储在服务A对应的数据库中。
从上述描述过程可以发现:这个过程可能会造成服务A和服务B对应的数据库可能会造成数据库不一致的问题。比如先在B中存储成功,然后在A中存储失败,即会出现数据不一致。为此,程序中简单对A存储信息的代码,使用try-catch,如果A存储失败,则删除先前在B中存储的数据。过程可用下面的伪代码表示:
@Transaction(rollbackFor = Exception.class)
public void saveInfo(InfoDTO info) {
// 1. 发送info.schoolList到第三方服务B,由第三方服务B将相应信息存储到自己的数据库中
SchoolEntity schoolEntity = A.saveBSchool(info.schoolList)
log.info("存储数据到服务B")
//2. 获取B服务为保存成功的数据生成的id
String schoolId = schoolEntity.getSchoolId();
try {
2.将info.student存储到自己服务(服务A)的数据库中。
但为了防止存储数据到服务B对应的数据库中成功,而存储数据到服务A的数据库失败
(比如未做参数校验时,输入参数长度大于数据库字段的最大长度)
导致的两个服务的数据库不一致的问题,这里用try对服务A存储信息进行捕获,发生异常,
删除步骤1中在B数据库中存储的那条数据
log.info("开始存储数据到服务A")
info.student.setSchoolId = schoolId;
A.saveStudent(info.student)
log.info("结束存储数据到服务A")
} catch (Exception e) {
log.info("存储数据到服务A发生异常")
A.deleteBSchool(info.schoolList)
}
由于我们需要在本服务A中获取服务B存储实体生成的id,因此不能改变两个服务存储数据的顺序。
如果A服务将请求存储到B服务中失败,A服务进行回滚。
如果A服务将请求存储到B服务中失成功, A服务将信息存储到自己数据库失败。调用A.deleteBSchool(info.schoolList),来删掉先前在服务B中存储的数据。
我们期望这样做来保证数据一致。但在实际运行程序中。我们存储一个数据会超过服务A对应数据库字段的最大长度(这里认为会引发代码 A.saveStudent(info.student) 的异常)。
但实际的后台打印却如下所示:
2021-04-23 19:37:49.640 INFO 43784 --- [io-23333-exec-2] c.t.n.s.a.impl.FormRequestServiceImpl : 存储数据到服务B
2021-04-23 19:37:49.640 INFO 43784 --- [io-23333-exec-2] c.t.n.s.a.impl.FormRequestServiceImpl : 开始存储数据到服务A
Hibernate: select databaseex0_.form_id as form_id1_12_0_, databaseex0_.ctime as ctime2_12_0_, databaseex0_.mtime as mtime3_12_0_, databaseex0_.applicant_name as applican4_12_0_, databaseex0_.email as email5_12_0_, databaseex0_.form_kind as form_kin6_12_0_, databaseex0_.form_title as form_tit7_12_0_, databaseex0_.execute_status as execute_8_12_0_, databaseex0_.execute_time as execute_9_12_0_, databaseex0_.executor_email as executo10_12_0_, databaseex0_.executor_name as executo11_12_0_, databaseex0_.form_status as form_st12_12_0_, databaseex0_.operation_type as operati13_12_0_ from database_execute databaseex0_ where databaseex0_.form_id=?
Hibernate: select databasere0_.form_id as form_id1_14_0_, databasere0_.ctime as ctime2_14_0_, databasere0_.mtime as mtime3_14_0_, databasere0_.applicant_name as applican4_14_0_, databasere0_.email as email5_14_0_, databasere0_.form_kind as form_kin6_14_0_, databasere0_.form_title as form_tit7_14_0_, databasere0_.command as command8_14_0_, databasere0_.comment as comment9_14_0_, databasere0_.database_name as databas10_14_0_, databasere0_.database_type as databas11_14_0_, databasere0_.environment as environ12_14_0_, databasere0_.form_status as form_st13_14_0_, databasere0_.instance_name as instanc14_14_0_, databasere0_.operation_type as operati15_14_0_, databasere0_.region as region16_14_0_, databasere0_.struct_type as struct_17_14_0_, databasere0_.table_name as table_n18_14_0_ from database_request databasere0_ where databasere0_.form_id=?
2021-04-23 19:37:49.704 INFO 43784 --- [io-23333-exec-2] c.t.n.s.a.impl.FormRequestServiceImpl : 结束存储数据到服务A
Hibernate: insert into database_execute (ctime, mtime, applicant_name, email, form_kind, form_title, execute_status, execute_time, executor_email, executor_name, form_status, operation_type, form_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Hibernate: insert into database_request (ctime, mtime, applicant_name, email, form_kind, form_title, command, comment, database_name, database_type, environment, form_status, instance_name, operation_type, region, struct_type, table_name, form_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
2021-04-23 19:37:49.732 WARN 43784 --- [io-23333-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 1406, SQLState: 22001
2021-04-23 19:37:49.732 ERROR 43784 --- [io-23333-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : Data truncation: Data too long for column 'command' at row 1
2021-04-23 19:37:49.736 ERROR 43784 --- [io-23333-exec-2] o.h.i.ExceptionMapperStandardImpl : HHH000346: Error during managed flush [org.hibernate.exception.DataException: could not execute statement]
可以看到,A服务存储数据到自己库的过程中发生异常,并未触发try-catch的异常,而去删除先前在服务B中存储的数据。
从后台打印的数据库语句来看。自己分析是,调用A.saveStudent(info.student)时,jpa并没有马上把数据实体存储到数据库中,而是先存储在缓存中,然后正常执行了业务(因为没将缓存中的数据刷到数据库,因此不会出发try-catch的异常捕获机制),在提交事务时,尝试刷新缓存中的数据到数据库,发生异常,而此时,由于业务逻辑中的try-catch已经在先前执行完,因此不会触发删除B先前存储的数据的操作,造成数据不一致问题。
二、save过程
Hibernate缓存包括两大类:一级缓存和二级缓存。
一级缓存又称为“Session的缓存”,它是内置的,不能被卸载(不能被卸载的意思就是这种缓存不具有可选性,必须有的功能,不可以取消session缓存)。由于Session对象的生命周期通常对应一个数据库事务或者一个应用事务,因此它的缓存是事务范围的缓存在第一级缓存中,持久化类的每个实例都具有唯一的OID。我们使用@Transactional 注解时,JpaTransactionManager会在开启事务前打开一个session,将事务绑定在这个session上,事务结束session关闭,所以后续内容将以粗略以事务作为一级缓存的生存时段。
二级缓存又称为“SessionFactory的缓存”,由于SessionFactory对象的生命周期和应用程序的整个过程对应,因此二级缓存是进程范围或者集群范围的缓存,有可能出现并发问题,因此需要采用适当的并发访问策略。第二级缓存是可选的,是一个可配置的插件,在默认情况下,SessionFactory不会启用这个插件,二级缓存应用场景局限性比较大,适用于数据要求的实时性和准确性不高、变动很少的情况。
我个人理解:一般不会开启二级缓存,会引入诸如并发方面的问题
我们使用CrudRepository.save方法保存或更新对象的流程如下:
从这张流程图:我们印证了对于第一节中问题的分析。
这里注意我们如果不给方法加事务(关于Spring事务,后面会进行总结,以伪程序而言,可以简单去除@Transaction来试验),再次执行程序,发现正常执行,具体输出如下 :
2021-04-23 19:59:26.798 INFO 43160 --- [io-23333-exec-3] c.t.n.s.a.impl.FormRequestServiceImpl : 存储数据到服务B
2021-04-23 19:59:26.798 INFO 43160 --- [io-23333-exec-3] c.t.n.s.a.impl.FormRequestServiceImpl : 开始存储数据到服务A
Hibernate: select databaseex0_.form_id as form_id1_12_0_, databaseex0_.ctime as ctime2_12_0_, databaseex0_.mtime as mtime3_12_0_, databaseex0_.applicant_name as applican4_12_0_, databaseex0_.email as email5_12_0_, databaseex0_.form_kind as form_kin6_12_0_, databaseex0_.form_title as form_tit7_12_0_, databaseex0_.execute_status as execute_8_12_0_, databaseex0_.execute_time as execute_9_12_0_, databaseex0_.executor_email as executo10_12_0_, databaseex0_.executor_name as executo11_12_0_, databaseex0_.form_status as form_st12_12_0_, databaseex0_.operation_type as operati13_12_0_ from database_execute databaseex0_ where databaseex0_.form_id=?
Hibernate: insert into database_execute (ctime, mtime, applicant_name, email, form_kind, form_title, execute_status, execute_time, executor_email, executor_name, form_status, operation_type, form_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Hibernate: select databasere0_.form_id as form_id1_14_0_, databasere0_.ctime as ctime2_14_0_, databasere0_.mtime as mtime3_14_0_, databasere0_.applicant_name as applican4_14_0_, databasere0_.email as email5_14_0_, databasere0_.form_kind as form_kin6_14_0_, databasere0_.form_title as form_tit7_14_0_, databasere0_.command as command8_14_0_, databasere0_.comment as comment9_14_0_, databasere0_.database_name as databas10_14_0_, databasere0_.database_type as databas11_14_0_, databasere0_.environment as environ12_14_0_, databasere0_.form_status as form_st13_14_0_, databasere0_.instance_name as instanc14_14_0_, databasere0_.operation_type as operati15_14_0_, databasere0_.region as region16_14_0_, databasere0_.struct_type as struct_17_14_0_, databasere0_.table_name as table_n18_14_0_ from database_request databasere0_ where databasere0_.form_id=?
Hibernate: insert into database_request (ctime, mtime, applicant_name, email, form_kind, form_title, command, comment, database_name, database_type, environment, form_status, instance_name, operation_type, region, struct_type, table_name, form_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
2021-04-23 19:59:26.887 WARN 43160 --- [io-23333-exec-3] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 1406, SQLState: 22001
2021-04-23 19:59:26.888 ERROR 43160 --- [io-23333-exec-3] o.h.engine.jdbc.spi.SqlExceptionHelper : Data truncation: Data too long for column 'command' at row 1
2021-04-23 19:59:26.891 ERROR 43160 --- [io-23333-exec-3] o.h.i.ExceptionMapperStandardImpl : HHH000346: Error during managed flush [org.hibernate.exception.DataException: could not execute statement]
2021-04-23 19:59:26.915 INFO 43160 --- [io-23333-exec-3] o.h.h.i.QueryTranslatorFactoryInitiator : HHH000397: Using ASTQueryTranslatorFactory
Hibernate: select databaseex0_.form_id as form_id1_12_, databaseex0_.ctime as ctime2_12_, databaseex0_.mtime as mtime3_12_, databaseex0_.applicant_name as applican4_12_, databaseex0_.email as email5_12_, databaseex0_.form_kind as form_kin6_12_, databaseex0_.form_title as form_tit7_12_, databaseex0_.execute_status as execute_8_12_, databaseex0_.execute_time as execute_9_12_, databaseex0_.executor_email as executo10_12_, databaseex0_.executor_name as executo11_12_, databaseex0_.form_status as form_st12_12_, databaseex0_.operation_type as operati13_12_ from database_execute databaseex0_ where databaseex0_.form_id=?
Hibernate: delete from database_execute where form_id=?
2021-04-23 19:59:27.044 ERROR 43160 --- [io-23333-exec-3] c.t.n.s.o.DatabaseRequestFormOperation : Save database form to request fail, error mssage: could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.DataException: could not execute statement
2021-04-23 19:59:27.044 INFO 43160 --- [io-23333-exec-3] c.t.n.s.a.impl.FormRequestServiceImpl : 存储数据到服务A发生异常
从输出打印看到,不给方法加事务的情况下,调用save方法,即保存数据到缓存,并马上将缓存中的数据刷新到数据库,然后再执行具体业务逻辑。注意insert语句和“Error during managed flush”出现的位置。
这里还需要注意:在执行save操作时,什么情况会只打一条insert语句。什么时候会先打一条select语句,再打一条insert语句。可以参考这篇博客:
其中如果entity实体设置了自增主键,就只会打印出一条insert语句,这种情况下即使给方法加了事务也不会出现上述问题)
而如果entity只是设置了主键,未设置增长策略,在调用save方法时,就会先打出一条select语句,再打出一条insert语句。如果加了事务,调用save方法时,会出现上述问题。
具体原因不清楚,需要看源码解决。
这些情况会在下面的问题复现中复现
三、问题复现
①使用自增主键,无论是否加事务,都先将缓存中的数据插入数据库,再执行业务操作,可以正常触发try-catch的异常捕获。
新建类MyFormTestEntity:
@Entity
@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class MyFormTestEntity {
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String name;
private String password;
}
新建类JPATestService和JPATestServiceImpl
public interface JPATestService {
MyFormTestEntity saveMyFormTest(MyFormTestEntity myFormTestEntity);
@Service
@Slf4j
public class JPATestServiceImpl implements JPATestService {
@Autowired
private MyFormTestRepository myFormTestRepository;
@Transactional(rollbackFor = Exception.class)
@Override
public MyFormTestEntity saveMyFormTest(MyFormTestEntity myFormTestEntity) {
MyFormTestEntity entity1 = null;
try {
log.info("111");
entity1 = myFormTestRepository.save(myFormTestEntity);
log.info("222");
} catch (Exception e) {
log.info("333");
return entity1;
controller方法(记得在controller类中@Autowired注入jpaTestService:
@PostMapping("/jpatest14")
public MyFormTestEntity jpatest14(@RequestBody MyFormTestEntity entity) {
return jpaTestService.saveMyFormTest(entity);
}
访问接口后台打印如下:
传参为:(其中password参数长度会超过数据库对应字段长度)
打印日志为:
2021-04-24 13:02:18.462 INFO 32036 --- [nio-7076-exec-2] c.e.t.e.j.s.impl.JPATestServiceImpl : 111
Hibernate: insert into my_form_test_entity (name, password) values (?, ?)
2021-04-24 13:02:18.510 WARN 32036 --- [nio-7076-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 1406, SQLState: 22001
2021-04-24 13:02:18.510 ERROR 32036 --- [nio-7076-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : Data truncation: Data too long for column 'password' at row 1
2021-04-24 13:02:18.515 INFO 32036 --- [nio-7076-exec-2] c.e.t.e.j.s.impl.JPATestServiceImpl : 333
可以看到使用自增主键,调用save方法只会打印出一条insert语句。并且插入数据后,再执行余下的业务逻辑。(不加事务,结果相同)
②不设置主键的自增策略,加事务,会先将数据保存在缓存,然后执行余下的业务逻辑,然后再将缓存中的数据刷新到数据库中,在刷数据到库时发生异常,但由于此时业务逻辑已经执行完,无法触发try-catch的异常捕获机制。
修改MyFormTestEntity如下:(去掉了@GeneratedValue(strategy = GenerationType.IDENTITY)
@Entity
@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class MyFormTestEntity {
private Long id;
private String name;
private String password;
}
其余不变,访问接口,传参如下:(其中password参数长度会超过数据库对应字段长度)
后台打印如下:
2021-04-24 13:17:27.975 INFO 42244 --- [nio-7076-exec-2] c.e.t.e.j.s.impl.JPATestServiceImpl : 111
Hibernate: select myformtest0_.id as id1_9_0_, myformtest0_.name as name2_9_0_, myformtest0_.password as password3_9_0_ from my_form_test_entity myformtest0_ where myformtest0_.id=?
2021-04-24 13:17:28.068 INFO 42244 --- [nio-7076-exec-2] c.e.t.e.j.s.impl.JPATestServiceImpl : 222
Hibernate: insert into my_form_test_entity (name, password, id) values (?, ?, ?)
2021-04-24 13:17:28.079 WARN 42244 --- [nio-7076-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 1406, SQLState: 22001
2021-04-24 13:17:28.079 ERROR 42244 --- [nio-7076-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : Data truncation: Data too long for column 'password' at row 1
2021-04-24 13:17:28.083 ERROR 42244 --- [nio-7076-exec-2] o.h.i.ExceptionMapperStandardImpl : HHH000346: Error during managed flush [org.hibernate.exception.DataException: could not execute statement]
当不给方法加事务时,即修改JPATestServiceImpl的saveMyFormTest方法如下:
@Override
public MyFormTestEntity saveMyFormTest(MyFormTestEntity myFormTestEntity) {
MyFormTestEntity entity1 = null;
try {
log.info("111");
entity1 = myFormTestRepository.save(myFormTestEntity);
log.info("222");
} catch (Exception e) {
log.info("333");
return entity1;
}
相同传参,再次访问接口,结果如下:
021-04-24 13:26:51.072 INFO 29972 --- [nio-7076-exec-3] c.e.t.e.j.s.impl.JPATestServiceImpl : 111