前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >聊聊jedis连接池参数配置

聊聊jedis连接池参数配置

作者头像
code4it
发布2023-09-25 21:03:52
4840
发布2023-09-25 21:03:52
举报
文章被收录于专栏:码匠的流水账码匠的流水账

本文主要研究一下jedis连接池的参数配置

JedisConfig

redis/clients/jedis/JedisPoolConfig.java

代码语言:javascript
复制
public class JedisPoolConfig extends GenericObjectPoolConfig<Jedis> {
  public JedisPoolConfig() {
    // defaults to make your life with connection pool easier :)
    setTestWhileIdle(true);
    setMinEvictableIdleTimeMillis(60000);
    setTimeBetweenEvictionRunsMillis(30000);
    setNumTestsPerEvictionRun(-1);
  }
}

JedisPoolConfig继承了GenericObjectPoolConfig,同时默认配置了testWhileIdle为true(默认为false),minEvictableIdleTime为60s(默认为30分钟),timeBetweenEvictionRuns为30s(默认为-1),numTestsPerEvictionRun为-1(即检测所有空闲连接,默认值为3)

GenericObjectPoolConfig

org/apache/commons/pool2/impl/GenericObjectPoolConfig.java

代码语言:javascript
复制
public class GenericObjectPoolConfig<T> extends BaseObjectPoolConfig<T> {
    /**
     * The default value for the {@code maxTotal} configuration attribute.
     * @see GenericObjectPool#getMaxTotal()
     */
    public static final int DEFAULT_MAX_TOTAL = 8;

    /**
     * The default value for the {@code maxIdle} configuration attribute.
     * @see GenericObjectPool#getMaxIdle()
     */
    public static final int DEFAULT_MAX_IDLE = 8;

    /**
     * The default value for the {@code minIdle} configuration attribute.
     * @see GenericObjectPool#getMinIdle()
     */
    public static final int DEFAULT_MIN_IDLE = 0;


    private int maxTotal = DEFAULT_MAX_TOTAL;

    private int maxIdle = DEFAULT_MAX_IDLE;

    private int minIdle = DEFAULT_MIN_IDLE;
}

GenericObjectPoolConfig继承了BaseObjectPoolConfig,其maxTotal为8,maxIdle为8,minIdle为0

BaseObjectPoolConfig

org/apache/commons/pool2/impl/BaseObjectPoolConfig.java

代码语言:javascript
复制
public abstract class BaseObjectPoolConfig<T> extends BaseObject implements Cloneable {
    private boolean lifo = DEFAULT_LIFO;
    public static final boolean DEFAULT_LIFO = true;

    private boolean fairness = DEFAULT_FAIRNESS;
    public static final boolean DEFAULT_FAIRNESS = false;

    private Duration maxWaitDuration = DEFAULT_MAX_WAIT;
    public static final Duration DEFAULT_MAX_WAIT = Duration.ofMillis(-1L);

    private Duration minEvictableIdleDuration = DEFAULT_MIN_EVICTABLE_IDLE_TIME;
    public static final Duration DEFAULT_MIN_EVICTABLE_IDLE_TIME =
            Duration.ofMillis(1000L * 60L * 30L);


    private Duration evictorShutdownTimeoutDuration = DEFAULT_EVICTOR_SHUTDOWN_TIMEOUT;
    public static final Duration DEFAULT_EVICTOR_SHUTDOWN_TIMEOUT =
            Duration.ofMillis(10L * 1000L);

    private Duration softMinEvictableIdleDuration = DEFAULT_SOFT_MIN_EVICTABLE_IDLE_TIME;
    public static final Duration DEFAULT_SOFT_MIN_EVICTABLE_IDLE_TIME =
            Duration.ofMillis(-1);

    private int numTestsPerEvictionRun = DEFAULT_NUM_TESTS_PER_EVICTION_RUN;
    public static final int DEFAULT_NUM_TESTS_PER_EVICTION_RUN = 3;

    private EvictionPolicy<T> evictionPolicy; // Only 2.6.0 applications set this

    private String evictionPolicyClassName = DEFAULT_EVICTION_POLICY_CLASS_NAME;
    public static final String DEFAULT_EVICTION_POLICY_CLASS_NAME = DefaultEvictionPolicy.class.getName();

    private boolean testOnCreate = DEFAULT_TEST_ON_CREATE;
    public static final boolean DEFAULT_TEST_ON_CREATE = false;

    private boolean testOnBorrow = DEFAULT_TEST_ON_BORROW;
    public static final boolean DEFAULT_TEST_ON_BORROW = false;

    private boolean testOnReturn = DEFAULT_TEST_ON_RETURN;
    public static final boolean DEFAULT_TEST_ON_RETURN = false;

    private boolean testWhileIdle = DEFAULT_TEST_WHILE_IDLE;
    public static final boolean DEFAULT_TEST_WHILE_IDLE = false;

    private Duration durationBetweenEvictionRuns = DEFAULT_TIME_BETWEEN_EVICTION_RUNS;
    public static final Duration DEFAULT_TIME_BETWEEN_EVICTION_RUNS = Duration
            .ofMillis(-1L);

    private boolean blockWhenExhausted = DEFAULT_BLOCK_WHEN_EXHAUSTED;
    public static final boolean DEFAULT_BLOCK_WHEN_EXHAUSTED = true;

    private boolean jmxEnabled = DEFAULT_JMX_ENABLE;
    public static final boolean DEFAULT_JMX_ENABLE = true;

    // TODO Consider changing this to a single property for 3.x
    private String jmxNamePrefix = DEFAULT_JMX_NAME_PREFIX;
    public static final String DEFAULT_JMX_NAME_PREFIX = "pool";

    private String jmxNameBase = DEFAULT_JMX_NAME_BASE;
    public static final String DEFAULT_JMX_NAME_BASE = null;
}

BaseObjectPoolConfig定义了lifo(默认为true)、fairness(默认为false)、maxWaitDuration(默认为-1)、minEvictableIdleDuration(默认为30分钟)、evictorShutdownTimeoutDuration(默认为10s)、softMinEvictableIdleDuration(默认为-1)、numTestsPerEvictionRun(默认为3)、evictionPolicyClassName(默认为DefaultEvictionPolicy.class.getName())、testOnCreate(默认为false)、testOnBorrow(默认为false)、testOnReturn(默认为false)、testWhileIdle(默认为false)、durationBetweenEvictionRuns(默认为-1)、blockWhenExhausted(默认为true)、jmxEnabled(默认为true)、jmxNamePrefix(默认为pool)、jmxNameBase(默认为null)

maxWaitDuration

org/apache/commons/pool2/impl/GenericObjectPool.java

代码语言:javascript
复制
			if (blockWhenExhausted) {
                if (p == null) {
                    if (borrowMaxWaitDuration.isNegative()) {
                        p = idleObjects.takeFirst();
                    } else {
                        p = idleObjects.pollFirst(borrowMaxWaitDuration);
                    }
                }
                if (p == null) {
                    throw new NoSuchElementException(appendStats(
                            "Timeout waiting for idle object, borrowMaxWaitDuration=" + borrowMaxWaitDuration));
                }
            }

borrow的时候在blockWhenExhausted为true时使用,-1的话则使用takeFirst方法,否则使用pollFirst(borrowMaxWaitDuration)方法

evict

getNumTests

org/apache/commons/pool2/impl/GenericObjectPool.java

代码语言:javascript
复制
    /**
     * Calculates the number of objects to test in a run of the idle object
     * evictor.
     *
     * @return The number of objects to test for validity
     */
    private int getNumTests() {
        final int numTestsPerEvictionRun = getNumTestsPerEvictionRun();
        if (numTestsPerEvictionRun >= 0) {
            return Math.min(numTestsPerEvictionRun, idleObjects.size());
        }
        return (int) (Math.ceil(idleObjects.size() /
                Math.abs((double) numTestsPerEvictionRun)));
    }

当numTestsPerEvictionRun为-1时,返回的是idleObjects.size()

EvictionConfig

org/apache/commons/pool2/impl/EvictionConfig.java

代码语言:javascript
复制
public class EvictionConfig {

    private static final Duration MAX_DURATION = Duration.ofMillis(Long.MAX_VALUE);
    private final Duration idleEvictDuration;
    private final Duration idleSoftEvictDuration;
    private final int minIdle;

    /**
     * Creates a new eviction configuration with the specified parameters.
     * Instances are immutable.
     *
     * @param idleEvictDuration Expected to be provided by
     *        {@link BaseGenericObjectPool#getMinEvictableIdleDuration()}
     * @param idleSoftEvictDuration Expected to be provided by
     *        {@link BaseGenericObjectPool#getSoftMinEvictableIdleDuration()}
     * @param minIdle Expected to be provided by
     *        {@link GenericObjectPool#getMinIdle()} or
     *        {@link GenericKeyedObjectPool#getMinIdlePerKey()}
     * @since 2.10.0
     */
    public EvictionConfig(final Duration idleEvictDuration, final Duration idleSoftEvictDuration, final int minIdle) {
        this.idleEvictDuration = PoolImplUtils.isPositive(idleEvictDuration) ? idleEvictDuration : MAX_DURATION;
        this.idleSoftEvictDuration = PoolImplUtils.isPositive(idleSoftEvictDuration) ? idleSoftEvictDuration : MAX_DURATION;
        this.minIdle = minIdle;
    }

    //......
}    

evict方法会构造EvictionConfig,minEvictableIdleDuration默认为30分钟;这里主要是针对负数进行了判断,负数则取的Long.MAX_VALUE;即softMinEvictableIdleDuration默认为-1,转换过来就是Long.MAX_VALUE

代码语言:javascript
复制
final EvictionConfig evictionConfig = new EvictionConfig(
                        getMinEvictableIdleDuration(),
                        getSoftMinEvictableIdleDuration(),
                        getMinIdle());

DefaultEvictionPolicy

org/apache/commons/pool2/impl/DefaultEvictionPolicy.java

代码语言:javascript
复制
public class DefaultEvictionPolicy<T> implements EvictionPolicy<T> {

    @Override
    public boolean evict(final EvictionConfig config, final PooledObject<T> underTest, final int idleCount) {
        // @formatter:off
        return (config.getIdleSoftEvictDuration().compareTo(underTest.getIdleDuration()) < 0 &&
                config.getMinIdle() < idleCount) ||
                config.getIdleEvictDuration().compareTo(underTest.getIdleDuration()) < 0;
        // @formatter:on
    }
}

DefaultEvictionPolicy的evict方法在idleCount大于minIdle且idleDuration大于idleSoftEvictDuration(默认为Long.MAX_VALUE)返回true,或者obj的idleDuration大于minEvictableIdleDuration(默认为30分钟)返回true

durationBetweenEvictionRuns

org/apache/commons/pool2/impl/BaseGenericObjectPool.java

代码语言:javascript
复制
    public final void setTimeBetweenEvictionRuns(final Duration timeBetweenEvictionRuns) {
        this.durationBetweenEvictionRuns = PoolImplUtils.nonNull(timeBetweenEvictionRuns, BaseObjectPoolConfig.DEFAULT_TIME_BETWEEN_EVICTION_RUNS);
        startEvictor(this.durationBetweenEvictionRuns);
    }

    final void startEvictor(final Duration delay) {
        synchronized (evictionLock) {
            final boolean isPositiverDelay = PoolImplUtils.isPositive(delay);
            if (evictor == null) { // Starting evictor for the first time or after a cancel
                if (isPositiverDelay) { // Starting new evictor
                    evictor = new Evictor();
                    EvictionTimer.schedule(evictor, delay, delay);
                }
            } else if (isPositiverDelay) { // Stop or restart of existing evictor: Restart
                synchronized (EvictionTimer.class) { // Ensure no cancel can happen between cancel / schedule calls
                    EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, true);
                    evictor = null;
                    evictionIterator = null;
                    evictor = new Evictor();
                    EvictionTimer.schedule(evictor, delay, delay);
                }
            } else { // Stopping evictor
                EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, false);
            }
        }
    }

setTimeBetweenEvictionRuns方法会durationBetweenEvictionRuns(默认为-1)去startEvictor;当改值为负数时则执行EvictionTimer.cancel,即不启动evictor线程

cancel

org/apache/commons/pool2/impl/EvictionTimer.java

代码语言:javascript
复制
    /**
     * Removes the specified eviction task from the timer.
     *
     * @param evictor   Task to be cancelled.
     * @param timeout   If the associated executor is no longer required, how
     *                  long should this thread wait for the executor to
     *                  terminate?
     * @param restarting The state of the evictor.
     */
    static synchronized void cancel(final BaseGenericObjectPool<?>.Evictor evictor, final Duration timeout,
            final boolean restarting) {
        if (evictor != null) {
            evictor.cancel();
            remove(evictor);
        }
        if (!restarting && executor != null && taskMap.isEmpty()) {
            executor.shutdown();
            try {
                executor.awaitTermination(timeout.toMillis(), TimeUnit.MILLISECONDS);
            } catch (final InterruptedException e) {
                // Swallow
                // Significant API changes would be required to propagate this
            }
            executor.setCorePoolSize(0);
            executor = null;
        }
    }

evictorShutdownTimeoutDuration即executor.awaitTermination的timeout参数

小结

参数类别

参数值

jedisConfig值

commons-pool默认值

解释

基本

lifo

true

true

后进先出

基本

fairness

false

false

非公平机制

数量

maxTotal

8

-

最大连接数

数量

maxIdle

8

-

最大空间连接数

数量

minIdle

0

-

最少空闲连接数

阻塞获取

blockWhenExhausted

true

true

连接池耗尽时,获取连接是否阻塞等待

阻塞获取

maxWaitMillis

-1

-1

blockWhenExhausted为true时阻塞等待多久

健康检测

testOnCreate

fasle

fasle

创建连接的时候检测

健康检测

testOnBorrow

false

false

借用连接的时候检测

健康检测

testOnReturn

false

false

归还连接的时候检测

健康检测

testWhileIdle

true

false

在空闲连接检测时检测

evict

durationBetweenEvictionRuns

30秒

-1

evictor线程运行间隔,-1为不运行

evict

minEvictableIdleDuration

60秒

30分钟

连接空闲的最小时间

evict

softMinEvictableIdleDuration

-1

-1

-1即Long.MAX_VALUE,它与idleCount>minIdle条件一起考虑

evict

numTestsPerEvictionRun

-1

3

每次空闲连接回收器线程(如果有)运行时检查的连接数量, -1表示全部

evict

evictionPolicyClassName

DefaultEvictionPolicy

DefaultEvictionPolicy

判断是否需要evict的PolicyClass

evict

evictorShutdownTimeoutDuration

10s

10s

关闭evictor线程池的等待时间

jmx

jmxEnabled

true

true

是否开启jmx

jmx

jmxNamePrefix

pool

pool

jmx名称前缀

jmx

jmxNameBase

null

null

null表示由pool自己定义jmxNameBase

JedisConfig默认帮我们配置了testWhileIdle为true(默认为false),minEvictableIdleTime为60s(默认为30分钟),timeBetweenEvictionRuns为30s(默认为-1),numTestsPerEvictionRun为-1(即检测所有空闲连接,默认值为3),利用evictor线程来检测空闲连接的健康情况

另外由聊聊jedis的return行为这篇分析可以得知在执行命令时若redis出问题,Jedis本身会标记底层connection为broken,在finally归还时会destory连接,保证连接池连接最终都会被清空重建。

另外在并发量比较大的场景,若要保证连接池的稳定数量则可以把minIdle设置成与maxTotal和maxIdle一样即可;若不想因为idle时间被频繁destory则可以设置minEvictableIdleTime为-1,evict方法始终返回false,evictor线程始终走的是testWhileIdle的逻辑;maxWaitMillis要设置一个合理值避免连接池耗尽阻塞线程,或者直接设置blockWhenExhausted为false

doc

本文参与?腾讯云自媒体分享计划,分享自微信公众号。
原始发表:2023-09-24 16:16,如有侵权请联系?cloudcommunity@tencent.com 删除

本文分享自 码匠的流水账 微信公众号,前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与?腾讯云自媒体分享计划? ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • JedisConfig
  • GenericObjectPoolConfig
  • BaseObjectPoolConfig
  • maxWaitDuration
  • evict
    • getNumTests
      • EvictionConfig
        • DefaultEvictionPolicy
          • durationBetweenEvictionRuns
            • cancel
            • 小结
            • doc
            相关产品与服务
            云数据库 Redis
            腾讯云数据库 Redis(TencentDB for Redis)是腾讯云打造的兼容 Redis 协议的缓存和存储服务。丰富的数据结构能帮助您完成不同类型的业务场景开发。支持主从热备,提供自动容灾切换、数据备份、故障迁移、实例监控、在线扩容、数据回档等全套的数据库服务。
            领券
            问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档
            http://www.vxiaotou.com