java - Spring Data Repository 多线程性能

Question

我遇到了多个线程访问同一个 spring 数据存储库对象的性能问题。许多线程在等待存储库对象的锁时被阻塞。大多数线程都对存储库对象执行相同的查询。当此线程 block 运行时，CPU 在所有内核上都已达到极限。有时它会下降，我认为这是来自等待存储库对象锁的阻塞线程。我已经通过分析验证了多个线程正在等待调用存储库对象中的相同方法。通过更改使用返回列表的方法的方法，我确实看到了性能的提升。但是锁定仍然是一个瓶颈。

更新:经过更多研究，我得出的结论是存储库对象是单例。当每个线程访问它时，这个对象被锁定。我如何制作存储库对象的原型(prototype)？ (我会为这个用例创建一个只读存储库。)配置是否必须更改？ Spring 数据是否已经这样做了？

MWE:

public interface EntityJpaRepository extends JpaRepository<Entity, Integer> {
    @Query(value = "select * from SomeTable where id = (?1);", nativeQuery = true)
    Entity findById(int id);

    //Method that returns a list of Entities
    @Query(value = "select * from SomeTable where id in (?1);", nativeQuery = true)
    List<Entity> findAllWithIds(List<Integer> ids);
}

@Component
@Scope("prototype")
public class AThread implements Runnable {
    @Autowired
    EntityJpaRepository myRepository;

    final int someId;         

    public AThread(int someId) {
      this.someId = someId;
    }

    @Override
    public void run() {
        //may call subMethod 1
        myRepository.findById(someId);
        //may call subMethod 1
        List<Integer> ids = someMethodWhichReturnsIDs();
        myRepository.findAllWithIds(ids);
        //may call subMethod 1
    }

    public void subMethod1(){
        //sometimes loop
        subMethod2();
        //may call 
    }
    public void subMethod2(){
        //more stuff
        List<Integer> ids = someMethodWhichReturnsIDs();
        //more stuff
    }
}

public static void main(String[] args){
ThreadPoolTaskExecutor taskExecutor = (ThreadPoolTaskExecutor) ctx.getBean("taskExecutor");

List<int> someInts;//assume this is full of ints.
for(int someId: someInts){
    taskExecutor.execute((Runnable)ctx.getBean("AThread", someId));
}
waitThreads(taskExecutor);

我会说我从我目前拥有的东西中获得了相当多的表现。我也不确定我是否已正确设置我的配置以使用多个线程/连接访问数据库。我不认为这是问题所在，但我提供了完整的配置。欢迎任何有关性能的提示。

@Configuration
@EnableJpaRepositories(basePackages= {"org.repository"})
@ContextConfiguration(locations={"classpath:META-INF/spring/app-context.xml"})
@ComponentScan(basePackages = "org.somepackage")
public class JpaConfiguration {

    @Value("#{mainDataSource}")
    private javax.sql.DataSource dataSource;

    @Bean
    public Map<String, Object> jpaProperties() {
        Map<String, Object> props = new HashMap<String, Object>();
        props.put("hibernate.dialect", MySQLDialect.class.getName());
        props.put("hibernate.format_sql", true);
        return props;
    }

    @Bean
    public JpaVendorAdapter jpaVendorAdapter() {
        HibernateJpaVendorAdapter hibernateJpaVendorAdapter = new HibernateJpaVendorAdapter();
        hibernateJpaVendorAdapter.setDatabase(Database.MYSQL);
        return hibernateJpaVendorAdapter;
    }

    @Bean
    public PlatformTransactionManager transactionManager() {
        return new JpaTransactionManager( entityManagerFactory().getObject());
    }

    @Bean
    public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
        LocalContainerEntityManagerFactoryBean lef = new LocalContainerEntityManagerFactoryBean();
        lef.setDataSource(this.dataSource);
        lef.setJpaPropertyMap(this.jpaProperties());
        lef.setJpaVendorAdapter(this.jpaVendorAdapter());
        return lef;
    }

    @Bean
    public ThreadPoolTaskExecutor taskExecutor() {
        ThreadPoolTaskExecutor pool = new ThreadPoolTaskExecutor();
        pool.setCorePoolSize(100);
        pool.setMaxPoolSize(500);
        pool.setWaitForTasksToCompleteOnShutdown(true);
        pool.setKeepAliveSeconds(1800);
        return pool;
    }
}

这是一个线程的堆栈跟踪，当我在它被阻塞时暂停进程/应用程序。我也有 pro-filer 的示例输出。当它运行更长的持续时间时，阻塞时间加起来。它显然被 @Autowired 存储库对象上的另一个线程阻塞。我以为我已经通过使用原型(prototype)作用域避免了这种情况。

taskExecutor-1  BLOCKED 4.68003 6320    34  2062    1   java.lang.Object@5842edfa   taskExecutor-9


java.lang.ClassLoader.loadClass(Unknown Source)
sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
java.lang.ClassLoader.loadClass(Unknown Source)
org.springframework.util.ClassUtils.isVisible(ClassUtils.java:1209)
org.springframework.util.ClassUtils.getAllInterfacesForClassAsSet(ClassUtils.java:1136)
org.springframework.util.ClassUtils.getAllInterfacesForClassAsSet(ClassUtils.java:1143)
org.springframework.util.ClassUtils.getAllInterfacesForClass(ClassUtils.java:1099)
org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:302)
com.sun.proxy.$Proxy41.createNativeQuery(Unknown Source)
org.springframework.data.jpa.repository.query.NativeJpaQuery.createJpaQuery(NativeJpaQuery.java:65)
org.springframework.data.jpa.repository.query.AbstractStringBasedJpaQuery.doCreateQuery(AbstractStringBasedJpaQuery.java:72)
org.springframework.data.jpa.repository.query.AbstractJpaQuery.createQuery(AbstractJpaQuery.java:165)
org.springframework.data.jpa.repository.query.JpaQueryExecution$SingleEntityExecution.doExecute(JpaQueryExecution.java:197)
org.springframework.data.jpa.repository.query.JpaQueryExecution.execute(JpaQueryExecution.java:74)
org.springframework.data.jpa.repository.query.AbstractJpaQuery.doExecute(AbstractJpaQuery.java:98)
org.springframework.data.jpa.repository.query.AbstractJpaQuery.execute(AbstractJpaQuery.java:89)
org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:421)
org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:381)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.data.repository.core.support.RepositoryFactorySupport$DefaultMethodInvokingMethodInterceptor.invoke(RepositoryFactorySupport.java:512)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodIntercceptor.invoke(    CrudMethodMetadataPostProcessor.java:122)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
com.sun.proxy.$Proxy81.findAllWithIds(Unknown Source)
sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
com.sun.proxy.$Proxy90.findAllWithIds(Unknown Source)
org.somepackage.AThread.subMethod2(AThread.java:696)
org.somepackage.AThread.subMethod1(AThread.java:346)
org.somepackage.AThread.run(AThread.java:132)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)

Answer 1

这看起来很像重复加载查询类的问题。 重复加载同一个类会导致对类加载器锁的激烈争用。 (此处示例:https://plumbr.eu/blog/locked-threads/classloading-and-locking)

sharedEntityManager 在第 302 行加载类 - 这将在 ClassLoader 中运行到第 404 行的同步块(synchronized block)