相关文章推荐
挂过科的山楂  ·  海泓道新橋七月二十七日啓用 西九龍公路往油麻 ...·  4 月前    · 
瘦瘦的山羊  ·  失眠睡不著怎麼辦?·  5 月前    · 
安静的黄豆  ·  Using Request objects ...·  5 月前    · 
帅气的煎饼  ·  Python风格规范 — Google ...·  6 月前    · 
奋斗的木瓜  ·  习近平同古巴共产党中央委员会第一书记、古巴国 ...·  1 年前    · 
Code  ›  聊聊GenericObjectPool的泄露检测 - 云+社区 -
https://cloud.tencent.com/developer/article/1340072
天涯
6 年前
菜单 腾讯云
备案 控制台
云+社区
  • 专栏
  • 问答
  • 沙龙
  • 快讯
  • 团队主页
  • 开发者手册
  • 智能钛AI
  • 在线学习中心
  • TVP
搜索
写文章 提问
登录 注册
展开

腾讯云·社区

登录

云+社区

  • 首页
  • 专栏
  • 问答
  • 沙龙
  • 快讯
  • 团队主页
  • 开发者手册
  • 智能钛AI
  • 在线学习中心
  • TVP
返回腾讯云官网

聊聊GenericObjectPool的泄露检测

写文章

聊聊GenericObjectPool的泄露检测

codecraft 发表于 码匠的流水账
169

腾讯云服务器 年付3折起

首次购买云服务器 最低3折起 超高性价比

在这篇文章中:

  • 序
  • object的状态
  • AbandonedConfig
  • 参数
  • removeAbandonedOnBorrow
  • removeAbandonedOnMaintenance
  • removeAbandonedTimeout
  • logAbandoned
  • useUsageTracking
  • doc

序

本文主要聊聊GenericObjectPool的abandon参数。主要用来做连接池的泄露检测用。

object的状态

commons-pool2-2.4.2-sources.jar!/org/apache/commons/pool2/PooledObjectState.java

public enum PooledObjectState {
     * In the queue, not in use.
    IDLE,
     * In use.
    ALLOCATED,
     * In the queue, currently being tested for possible eviction.
    EVICTION,
     * Not in the queue, currently being tested for possible eviction. An
     * attempt to borrow the object was made while being tested which removed it
     * from the queue. It should be returned to the head of the queue once
     * eviction testing completes.
     * TODO: Consider allocating object and ignoring the result of the eviction
     *       test.
    EVICTION_RETURN_TO_HEAD,
     * In the queue, currently being validated.
    VALIDATION,
     * Not in queue, currently being validated. The object was borrowed while
     * being validated and since testOnBorrow was configured, it was removed
     * from the queue and pre-allocated. It should be allocated once validation
     * completes.
    VALIDATION_PREALLOCATED,
     * Not in queue, currently being validated. An attempt to borrow the object
     * was made while previously being tested for eviction which removed it from
     * the queue. It should be returned to the head of the queue once validation
     * completes.
    VALIDATION_RETURN_TO_HEAD,
     * Failed maintenance (e.g. eviction test or validation) and will be / has
     * been destroyed
    INVALID,
     * Deemed abandoned, to be invalidated.
    ABANDONED,
     * Returning to the pool.
    RETURNING
}

abandon一般是用于连接泄露的检测,检测的是在使用的对象,比如怀疑那个对象被占用时间超长,那估计是程序异常或bug导致对象borrow了但忘记归还,或者对象borrow之后使用时间太长。

AbandonedConfig

除了commons-pool2-2.4.2-sources.jar!/org/apache/commons/pool2/impl/GenericObjectPoolConfig.java,还有这个AbandonedConfig commons-pool2-2.4.2-sources.jar!/org/apache/commons/pool2/impl/AbandonedConfig.java

public class AbandonedConfig {
     * Whether or not borrowObject performs abandoned object removal.
    private boolean removeAbandonedOnBorrow = false;
     * <p>Flag to remove abandoned objects if they exceed the
     * removeAbandonedTimeout when borrowObject is invoked.</p>
     * <p>The default value is false.</p>
     * <p>If set to true, abandoned objects are removed by borrowObject if
     * there are fewer than 2 idle objects available in the pool and
     * <code>getNumActive() &gt; getMaxTotal() - 3</code></p>
     * @return true if abandoned objects are to be removed by borrowObject
    public boolean getRemoveAbandonedOnBorrow() {
        return this.removeAbandonedOnBorrow;
     * <p>Flag to remove abandoned objects if they exceed the
     * removeAbandonedTimeout when borrowObject is invoked.</p>
     * @param removeAbandonedOnBorrow true means abandoned objects will be
     *   removed by borrowObject
     * @see #getRemoveAbandonedOnBorrow()
    public void setRemoveAbandonedOnBorrow(boolean removeAbandonedOnBorrow) {
        this.removeAbandonedOnBorrow = removeAbandonedOnBorrow;
     * Whether or not pool maintenance (evictor) performs abandoned object
     * removal.
    private boolean removeAbandonedOnMaintenance = false;
     * Timeout in seconds before an abandoned object can be removed.
    private int removeAbandonedTimeout = 300;
     * Determines whether or not to log stack traces for application code
     * which abandoned an object.
    private boolean logAbandoned = false;
     * PrintWriter to use to log information on abandoned objects.
     * Use of default system encoding is deliberate.
    private PrintWriter logWriter = new PrintWriter(System.out);
     * If the pool implements {@link UsageTracking}, should the pool record a
     * stack trace every time a method is called on a pooled object and retain
     * the most recent stack trace to aid debugging of abandoned objects?
    private boolean useUsageTracking = false;
}

参数

  • removeAbandonedOnBorrow 在borrow的时候,是否执行abandon判断,默认false
  • removeAbandonedOnMaintenance 是否在evictor中执行abandon判断,默认false
  • removeAbandonedTimeout 一个对象在被borrow之后多少秒未归还则认为是abandon,默认为300
  • logAbandoned 是否打印abandon的日志,默认为false
  • useUsageTracking 是否追踪对象调用并保留最近的调用记录方便debug

removeAbandonedOnBorrow

在borrow方法里头

public T borrowObject(long borrowMaxWaitMillis) throws Exception {
        assertOpen();
        AbandonedConfig ac = this.abandonedConfig;
        if (ac != null && ac.getRemoveAbandonedOnBorrow() &&
                (getNumIdle() < 2) &&
                (getNumActive() > getMaxTotal() - 3) ) {
            removeAbandoned(ac);
        PooledObject<T> p = null;
        // Get local copy of current config so it is consistent for entire
        // method execution
        boolean blockWhenExhausted = getBlockWhenExhausted();
        boolean create;
        long waitTime = System.currentTimeMillis();
        //......
        updateStatsBorrow(p, System.currentTimeMillis() - waitTime);
        return p.getObject();
    }

removeAbandonedOnMaintenance

在evictor线程里头

public void evict() throws Exception {
        assertOpen();
        if (idleObjects.size() > 0) {
            PooledObject<T> underTest = null;
            EvictionPolicy<T> evictionPolicy = getEvictionPolicy();
            synchronized (evictionLock) {
                EvictionConfig evictionConfig = new EvictionConfig(
                        getMinEvictableIdleTimeMillis(),
                        getSoftMinEvictableIdleTimeMillis(),
                        getMinIdle());
                boolean testWhileIdle = getTestWhileIdle();
                for (int i = 0, m = getNumTests(); i < m; i++) {
                    if (evictionIterator == null || !evictionIterator.hasNext()) {
                        evictionIterator = new EvictionIterator(idleObjects);
                    if (!evictionIterator.hasNext()) {
                        // Pool exhausted, nothing to do here
                        return;
                    try {
                        underTest = evictionIterator.next();
                    } catch (NoSuchElementException nsee) {
                        // Object was borrowed in another thread
                        // Don't count this as an eviction test so reduce i;
                        evictionIterator = null;
                        continue;
                    if (!underTest.startEvictionTest()) {
                        // Object was borrowed in another thread
                        // Don't count this as an eviction test so reduce i;
                        continue;
                    // User provided eviction policy could throw all sorts of
                    // crazy exceptions. Protect against such an exception
                    // killing the eviction thread.
                    boolean evict;
                    try {
                        evict = evictionPolicy.evict(evictionConfig, underTest,
                                idleObjects.size());
                    } catch (Throwable t) {
                        // Slightly convoluted as SwallowedExceptionListener
                        // uses Exception rather than Throwable
                        PoolUtils.checkRethrow(t);
                        swallowException(new Exception(t));
                        // Don't evict on error conditions
                        evict = false;
                    if (evict) {
                        destroy(underTest);
                        destroyedByEvictorCount.incrementAndGet();
                    } else {
                        if (testWhileIdle) {
                            boolean active = false;
                            try {
                                factory.activateObject(underTest);
                                active = true;
                            } catch (Exception e) {
                                destroy(underTest);
                                destroyedByEvictorCount.incrementAndGet();
                            if (active) {
                                if (!factory.validateObject(underTest)) {
                                    destroy(underTest);
                                    destroyedByEvictorCount.incrementAndGet();
                                } else {
                                    try {
                                        factory.passivateObject(underTest);
                                    } catch (Exception e) {
                                        destroy(underTest);
                                        destroyedByEvictorCount.incrementAndGet();
                        if (!underTest.endEvictionTest(idleObjects)) {
                            // TODO - May need to add code here once additional
                            // states are used
        AbandonedConfig ac = this.abandonedConfig;
        if (ac != null && ac.getRemoveAbandonedOnMaintenance()) {
            removeAbandoned(ac);
    }

removeAbandonedTimeout

在removeAbandoned方法里头

/**
     * Recover abandoned objects which have been checked out but
     * not used since longer than the removeAbandonedTimeout.
     * @param ac The configuration to use to identify abandoned objects
    private void removeAbandoned(AbandonedConfig ac) {
        // Generate a list of abandoned objects to remove
        final long now = System.currentTimeMillis();
        final long timeout =
                now - (ac.getRemoveAbandonedTimeout() * 1000L);
        ArrayList<PooledObject<T>> remove = new ArrayList<PooledObject<T>>();
        Iterator<PooledObject<T>> it = allObjects.values().iterator();
        while (it.hasNext()) {
            PooledObject<T> pooledObject = it.next();
            synchronized (pooledObject) {
                if (pooledObject.getState() == PooledObjectState.ALLOCATED &&
                        pooledObject.getLastUsedTime() <= timeout) {
                    pooledObject.markAbandoned();
                    remove.add(pooledObject);
        // Now remove the abandoned objects
        Iterator<PooledObject<T>> itr = remove.iterator();
        while (itr.hasNext()) {
            PooledObject<T> pooledObject = itr.next();
            if (ac.getLogAbandoned()) {
                pooledObject.printStackTrace(ac.getLogWriter());
            try {
                invalidateObject(pooledObject.getObject());
            } catch (Exception e) {
                e.printStackTrace();
    }

标记为abandon之后,立马放入remove队列中,然后遍历进行invalidateObject

public void invalidateObject(T obj) throws Exception {
        PooledObject<T> p = allObjects.get(new IdentityWrapper<T>(obj));
        if (p == null) {
            if (isAbandonedConfig()) {
                return;
            } else {
                throw new IllegalStateException(
                        "Invalidated object not currently part of this pool");
        synchronized (p) {
            if (p.getState() != PooledObjectState.INVALID) {
                destroy(p);
        ensureIdle(1, false);
    }

logAbandoned

最后是作用在这个类 commons-pool2-2.4.2-sources.jar!/org/apache/commons/pool2/impl/DefaultPooledObject.java

public synchronized boolean allocate() {
        if (state == PooledObjectState.IDLE) {
            state = PooledObjectState.ALLOCATED;
            lastBorrowTime = System.currentTimeMillis();
            lastUseTime = lastBorrowTime;
            borrowedCount++;
            if (logAbandoned) {
                borrowedBy = new AbandonedObjectCreatedException();
            return true;
        } else if (state == PooledObjectState.EVICTION) {
            // TODO Allocate anyway and ignore eviction test
            state = PooledObjectState.EVICTION_RETURN_TO_HEAD;
            return false;
        // TODO if validating and testOnBorrow == true then pre-allocate for
        // performance
        return false;
    }

useUsageTracking

public void use(T pooledObject) {
        AbandonedConfig ac = this.abandonedConfig;
        if (ac != null && ac.getUseUsageTracking()) {
            PooledObject<T> wrapper = allObjects.get(new IdentityWrapper<T>(pooledObject));
            wrapper.use();
    }

就是会调用一下PooledObject的use进行统计 commons-pool2-2.4.2-sources.jar!/org/apache/commons/pool2/proxy/BaseProxyHandler.java

    /**
     * Invoke the given method on the wrapped object.
     * @param method    The method to invoke
     * @param args      The arguments to the method
     * @return          The result of the method call
     * @throws Throwable    If the method invocation fails
    Object doInvoke(Method method, Object[] args) throws Throwable {
        validateProxiedObject();
        T object = getPooledObject();
        if (usageTracking != null) {
            usageTracking.use(object);
        return method.invoke(object, args);
    }

doc

  • commons-pool2 2 - poolObject API与状态机
  • 对象池common-pool2源码分析之对象状态
  • GenericObjectPool 避免泄漏
  • apache commons pool之GenericObjectPool分析(通用对象池技术)

原文发布于微信公众号 - 码匠的流水账(geek_luandun)

原文发表时间: 2017-10-19

本文参与 腾讯云自媒体分享计划 ,欢迎正在阅读的你也加入,一起分享。

发表于 2018-09-17 2018-09-17 15:15:10
其他
举报
1
分享
  • 分享文章到朋友圈

  • 分享文章到 QQ
  • 分享文章到微博
  • 复制文章链接到剪贴板

扫码关注云+社区

领取腾讯云代金券

码匠的流水账

904 篇文章 48 人订阅
  • 聊聊selenium的webdriver的超时参数
  • jodconverter4.1.0版本改进解析
  • futureTask的超时原理解析
  • 聊聊phantomjs的优化措施
  • 解决WebDriverWait中的cannot be applied的问题

我来说两句

0 条评论
登录 后参与评论
  • 上一篇:聊聊selenium的webdriver的超时参数
  • 下一篇:除了敲代码,程序员的一天日常是怎样的

相关文章

来自专栏 软件开发 -- 分享 互助 成长

为什么无返回值的链表的插入操作头结点一定要用指向指针的指针

前言: 为什么链表的插入操作头结点一定要用指向指针的指针?之前自己对这个问题总是一知半解,今天终于花了点时间彻底搞懂了。 总的来说这样做的目的是为了应对“空链表...

236 7
来自专栏 SDNLAB

《P4语言规范》Header & Instances详解

一、前言

 
推荐文章
挂过科的山楂  ·  海泓道新橋七月二十七日啓用 西九龍公路往油麻地支路八月三日起封閉
4 月前
瘦瘦的山羊  ·  失眠睡不著怎麼辦?
5 月前
安静的黄豆  ·  Using Request objects — Guzzle documentation
5 月前
帅气的煎饼  ·  Python风格规范 — Google 开源项目风格指南
6 月前
奋斗的木瓜  ·  习近平同古巴共产党中央委员会第一书记、古巴国家主席迪亚斯-卡内尔举行会谈
1 年前
今天看啥   ·   Py中国   ·   codingpro   ·   小百科   ·   link之家   ·   卧龙AI搜索
删除内容请联系邮箱 2879853325@qq.com
Code - 代码工具平台
© 2024 ~ 沪ICP备11025650号