Netty ObjectPool对象池技术原理分析

一、ObjectPool使用示例

1.对需要使用对象池的对象，定义一个ObjectPool的静态全局变量RECYCLE，用于对象的分配和回收。并在对象内定义一个ObjectPool.Handle成员变量，并且将此变量作为构造函数参数传入，并将构造函数作为私有。然后添加一个回收的方法Recycle,在不需要此对象时调用handle.recycle()

获取对象则调用ObjectPool.get

@Slf4j
public class ObjectRecycleTest {private static ExecutorService executor = new ThreadPoolExecutor(3, 6, 10,TimeUnit.SECONDS, new ArrayBlockingQueue<Runnable>(100), new NamedThreadFactory("local", false));public static final class People {String name;int age;boolean sex;private ObjectPool.Handle<People> handle;private static final ObjectPool<People> RECYCLE = ObjectPool.newPool((handle) -> {return new People(handle);});private People(ObjectPool.Handle<People> handle) {this.handle = handle;}public static People newInstance(String name, int age, boolean sex) {People people = RECYCLE.get();people.age = age;people.name = name;people.sex = sex;return people;}public void recycle() {name = "";age = 0;handle.recycle(this);}}public void testRecycle() {Set<People> peopleList = new HashSet<People>();for (int i = 0; i < 5; i++) {People p1 = People.newInstance("jack", 28, i % 2 == 0);
//            log.debug(" p1:{}",p1);peopleList.add(p1);}peopleList.stream().forEach(t -> {if (t.sex) {t.recycle();}});try {executor.submit(() -> {peopleList.stream().forEach(t -> {if (!t.sex) {t.recycle();}});}).get();} catch (InterruptedException e) {e.printStackTrace();} catch (ExecutionException e) {e.printStackTrace();}for (int i = 0; i < 2; i++) {People p2 = People.newInstance("jack", 28, i % 2 == 0);if (peopleList.contains(p2)) {log.debug(" in exist p2:{}", p2);} else {
//                log.debug(" new instance p2:{}",p2);}}executor.submit(() -> {People p3 = People.newInstance("qiuye", 26, true);log.debug(" p3:{}", p3);});}public static void main(String[] args) {ObjectRecycleTest objectRecycleTest = new ObjectRecycleTest();objectRecycleTest.testRecycle();}
}

二、实例对象创建流程

1.调用ObjectPool.get来获取实例,这是对Recycler的一个封装。

public abstract class ObjectPool<T> {ObjectPool() { }public static <T> ObjectPool<T> newPool(final ObjectCreator<T> creator) {return new RecyclerObjectPool<T>(ObjectUtil.checkNotNull(creator, "creator"));}private static final class RecyclerObjectPool<T> extends ObjectPool<T> {private final Recycler<T> recycler;RecyclerObjectPool(final ObjectCreator<T> creator) {recycler = new Recycler<T>() {@Overrideprotected T newObject(Handle<T> handle) {return creator.newObject(handle);}};}@Overridepublic T get() {return recycler.get();}}
}

2.Recycler的类中有一个threadLocal的stack类型的栈对象(保证每个线程数据私有和安全)，这里面保存有所有曾经回收到准备重利用的对象。

public abstract class Recycler<T> {private final FastThreadLocal<Stack<T>> threadLocal = new FastThreadLocal<Stack<T>>() {@Overrideprotected Stack<T> initialValue() {return new Stack<T>(Recycler.this, Thread.currentThread(), maxCapacityPerThread, maxSharedCapacityFactor,interval, maxDelayedQueuesPerThread, delayedQueueInterval);}@Overrideprotected void onRemoval(Stack<T> value) {// Let us remove the WeakOrderQueue from the WeakHashMap directly if its safe to remove some overheadif (value.threadRef.get() == Thread.currentThread()) {if (DELAYED_RECYCLED.isSet()) {DELAYED_RECYCLED.get().remove(value);}}}};

3.Recycler.get方法，就是从stack中获取对象，获取为空，则新建一个对象和Handle，并将Handle与Stack关联。

    public final T get() {if (maxCapacityPerThread == 0) {return newObject((Handle<T>) NOOP_HANDLE);}Stack<T> stack = threadLocal.get();DefaultHandle<T> handle = stack.pop();if (handle == null) {handle = stack.newHandle();handle.value = newObject(handle);}return (T) handle.value;}

4.stack.pop方法，则是获取历史的回收的对象。

private static final class Stack<T> {
DefaultHandle<?>[] elements;
private volatile WeakOrderQueue head;
DefaultHandle<T> pop() {int size = this.size;if (size == 0) {if (!scavenge()) {return null;}size = this.size;if (size <= 0) {// double check, avoid racesreturn null;}}size --;DefaultHandle ret = elements[size];elements[size] = null;this.size = size;if (ret.lastRecycledId != ret.recycleId) {throw new IllegalStateException("recycled multiple times");}ret.recycleId = 0;ret.lastRecycledId = 0;return ret;}

这里有两种对象来源。

（1）.elements数组是保存新建对象和回收对象在同一个线程的回收对象。

（2）.类型WeakOrderQueue的head对象则是一个弱引用链接队列，这个队列用来保存所有创建和回收线程不一致的回收对象。

(3).首先从elements获取回收对象，如果这里不为空，则返回第一个，总数减1.

(4).如果elements为空，则将head队列中的回收对象复制到elements，再次获取。

5.我们首先看一下WeakOrderQueue的数据结构

这里面有两个next属性

(1).本身的next成员变量，是指向下一个线程回收的WeakOrderQueue，WeakOrderQueue这个在一个stack中也是多个，是一个单身链接。

(2).本身有个head,tail节点，用来指向对象内部的Link对象队列。插入数据从tail进行，弹出数据从head进行。

  private static final class WeakOrderQueue extends WeakReference<Thread> {static final WeakOrderQueue DUMMY = new WeakOrderQueue();@SuppressWarnings("serial")static final class Link extends AtomicInteger {final DefaultHandle<?>[] elements = new DefaultHandle[LINK_CAPACITY];int readIndex;Link next;}private static final class Head {private final AtomicInteger availableSharedCapacity;Link link;Head(AtomicInteger availableSharedCapacity) {this.availableSharedCapacity = availableSharedCapacity;}private void reclaimSpace(int space) {availableSharedCapacity.addAndGet(space);}void relink(Link link) {reclaimSpace(LINK_CAPACITY);this.link = link;}}// chain of data itemsprivate final Head head;private Link tail;// pointer to another queue of delayed items for the same stackprivate WeakOrderQueue next;private final int id = ID_GENERATOR.getAndIncrement();private final int interval;private int handleRecycleCount;

5.从head队列复制回收对象列表到elements的流程，这里先找一个可用的不为空的队列，并且将回收对象(这里的回收对象都为Handle类型)列表复制到elements数组。

  private boolean scavengeSome() {WeakOrderQueue prev;WeakOrderQueue cursor = this.cursor;if (cursor == null) {prev = null;cursor = head;if (cursor == null) {return false;}} else {prev = this.prev;}boolean success = false;do {if (cursor.transfer(this)) {success = true;break;}WeakOrderQueue next = cursor.getNext();if (cursor.get() == null) {if (cursor.hasFinalData()) {for (;;) {if (cursor.transfer(this)) {success = true;} else {break;}}}if (prev != null) {// Ensure we reclaim all space before dropping the WeakOrderQueue to be GC'ed.cursor.reclaimAllSpaceAndUnlink();prev.setNext(next);}} else {prev = cursor;}cursor = next;} while (cursor != null && !success);this.prev = prev;this.cursor = cursor;return success;}

6.具体的从WeakOrderQueue复制数据到stack，是由WeakOrderQueue的transfer完成。

    boolean transfer(Stack<?> dst) {Link head = this.head.link;final int srcStart = head.readIndex;int srcEnd = head.get();final int srcSize = srcEnd - srcStart;if (srcSize == 0) {return false;}final int dstSize = dst.size;final int expectedCapacity = dstSize + srcSize;if (expectedCapacity > dst.elements.length) {final int actualCapacity = dst.increaseCapacity(expectedCapacity);srcEnd = min(srcStart + actualCapacity - dstSize, srcEnd);}if (srcStart != srcEnd) {final DefaultHandle[] srcElems = head.elements;final DefaultHandle[] dstElems = dst.elements;int newDstSize = dstSize;for (int i = srcStart; i < srcEnd; i++) {DefaultHandle<?> element = srcElems[i];if (element.recycleId == 0) {element.recycleId = element.lastRecycledId;} else if (element.recycleId != element.lastRecycledId) {throw new IllegalStateException("recycled already");}srcElems[i] = null;if (dst.dropHandle(element)) {// Drop the object.continue;}element.stack = dst;dstElems[newDstSize ++] = element;}head.readIndex = srcEnd;dst.size = newDstSize;return true;} else {// The destination stack is full already.return false;}}

这里面的代码比较长，也就是从head.link节点开始，复制handle对象数据到elements中。这里特殊要注意一下，这种跨线程的对象，也会采取丢失策略。即丢7取1.即stack.drophandle，到时在回收对象时讲述。

7.这里再回到第4步，由于stack.elements有数据了，所以跟之前的流程一样。从这里可以看出，同线程回收的对象，都可以重复使用，跨线程回收的对象，只能丢7取1.

三、实例对象回收过程。

1.从handle.recycle开始,handle保存在People实例对象的属性中。

  private static final class DefaultHandle<T> implements Recycler.Handle<T> {int lastRecycledId;int recycleId;boolean hasBeenRecycled;Recycler.Stack<?> stack;Object value;DefaultHandle(Recycler.Stack<?> stack) {this.stack = stack;}public void recycle(Object object) {if (object != this.value) {throw new IllegalArgumentException("object does not belong to handle");} else {Recycler.Stack<?> stack = this.stack;if (this.lastRecycledId == this.recycleId && stack != null) {stack.push(this);} }}}

2.这里会调用stack.push(this)，stack即为创建时当前线程所绑定的threadlocal对象。

Stack       void push(Recycler.DefaultHandle<?> item) {Thread currentThread = Thread.currentThread();if (this.threadRef.get() == currentThread) {this.pushNow(item);} else {this.pushLater(item, currentThread);}}

3.判断如果是当前线程，则pushNow,否则pushLater,pushNow就是放到elements数组，pushLater就是放到WeakOrderQueue类型的head队列。

我们先看pushNow

       private void pushNow(Recycler.DefaultHandle<?> item) {if ((item.recycleId | item.lastRecycledId) != 0) {throw new IllegalStateException("recycled already");} else {item.recycleId = item.lastRecycledId = Recycler.OWN_THREAD_ID;int size = this.size;if (size < this.maxCapacity && !this.dropHandle(item)) {if (size == this.elements.length) {this.elements = (Recycler.DefaultHandle[])Arrays.copyOf(this.elements, Math.min(size << 1, this.maxCapacity));}this.elements[size] = item;this.size = size + 1;}}}

这个比较简单，就是放到elements数组，并且 SIZE自增，但是要注意的是会进行一次dropHandle过滤，8个对象只有一个会存下来，其它的都会被GC回收，不能重复使用。

        boolean dropHandle(Recycler.DefaultHandle<?> handle) {if (!handle.hasBeenRecycled) {if (this.handleRecycleCount < this.interval) {++this.handleRecycleCount;return true;}this.handleRecycleCount = 0;handle.hasBeenRecycled = true;}return false;}

4.接下来我们看pushLater，这里面会判断当前stack对象是否在当前回收线程的

private static final FastThreadLocal<Map<Recycler.Stack<?>, Recycler.WeakOrderQueue>> DELAYED_RECYCLED这个LOCAL线程数据的Map中是否存在(一个线程可以有多个stack,多个对象回收器)。如果不存在，则创建一个newWeakOrderQueue，然后将当前handle保存到WeakOrderQueue中。

   private void pushLater(Recycler.DefaultHandle<?> item, Thread thread) {if (this.maxDelayedQueues != 0) {Map<Recycler.Stack<?>, Recycler.WeakOrderQueue> delayedRecycled = (Map)Recycler.DELAYED_RECYCLED.get();Recycler.WeakOrderQueue queue = (Recycler.WeakOrderQueue)delayedRecycled.get(this);if (queue == null) {if (delayedRecycled.size() >= this.maxDelayedQueues) {delayedRecycled.put(this, Recycler.WeakOrderQueue.DUMMY);return;}if ((queue = this.newWeakOrderQueue(thread)) == null) {return;}delayedRecycled.put(this, queue);} else if (queue == Recycler.WeakOrderQueue.DUMMY) {return;}queue.add(item);}}

5.新建队列,这里可以看到新建一个head,tail，并且head.link指向第一个tail，

        static Recycler.WeakOrderQueue newQueue(Recycler.Stack<?> stack, Thread thread) {if (!Recycler.WeakOrderQueue.Head.reserveSpaceForLink(stack.availableSharedCapacity)) {return null;} else {Recycler.WeakOrderQueue queue = new Recycler.WeakOrderQueue(stack, thread);stack.setHead(queue);return queue;}}private WeakOrderQueue(Recycler.Stack<?> stack, Thread thread) {super(thread);this.id = Recycler.ID_GENERATOR.getAndIncrement();this.tail = new Recycler.WeakOrderQueue.Link();this.head = new Recycler.WeakOrderQueue.Head(stack.availableSharedCapacity);this.head.link = this.tail;this.interval = stack.delayedQueueInterval;this.handleRecycleCount = this.interval;}

6.入队过程，就是放到tail.elements数组中，因为tail为Link类型，这个类又继承于

AtomicInteger，所以巧妙的运行AtomicInteger.value来标志为写位置。这个有个要注意，如果tail中的elements已满，则新建一个Link作为新的tail,老的tail的NEXT指向新tail节点，而head.link则一直指向第一个tail,构成一个单向链接。

        void add(Recycler.DefaultHandle<?> handle) {handle.lastRecycledId = this.id;if (this.handleRecycleCount < this.interval) {++this.handleRecycleCount;} else {this.handleRecycleCount = 0;Recycler.WeakOrderQueue.Link tail = this.tail;int writeIndex;if ((writeIndex = tail.get()) == Recycler.LINK_CAPACITY) {Recycler.WeakOrderQueue.Link link = this.head.newLink();if (link == null) {return;}this.tail = tail = tail.next = link;writeIndex = tail.get();}tail.elements[writeIndex] = handle;handle.stack = null;tail.lazySet(writeIndex + 1);}}

7.查看堆栈快照数据，先他截6个对象，然后当前线程回收3个，其它线程回收3个。则是在stack的elements中有一个和WeakOrderQueue的head中有一个（丢7取1策略）

8.至此，分析完成，可以看到，对象池不是完全重复使用，而是丢7存1，防止内存溢出。

Netty ObjectPool对象池技术原理分析相关推荐

java对象工厂池_[转载]Java对象池技术的原理及其实现
作者:汪永好出处:计算机与信息技术责任编辑: 方舟 [ 2006-11-25 07:00 ] 摘要 :本文在分析对象池技术基本原理的基础上,给出了对象池技术的两种实现方式.还指出了使用对象池技术 ...
java 对象池博客_Java对象池技术的原理及其实现的小结
一起学习 Java对象的生命周期大致包括三个阶段:对象的创建,对象的使用,对象的清除.因此,对象的生命周期长度可用如下的表达式表示:T = T1 T2 T3.其中T1表示对象的创建时间,T2表示对象的 ...
java 对象池实现_Java对象池技术的原理及其实现
看到一片有关于java 对象基础知识,故转载一下,同时学习一下. 摘要本文在分析对象池技术基本原理的基础上,给出了对象池技术的两种实现方式.还指出了使用对象池技术时所应注意的问题. 关键词对象池 ...
面试官问：对象池技术了解吗？apache common pool2呢？
欢迎关注方志朋的博客,回复"666"获面试宝典对象池顾名思义就是存放对象的池,与我们常听到的线程池.数据库连接池.http连接池等一样,都是典型的池化设计思想. 对象池的优点就是 ...
Unity中的资源管理-对象池技术(3)
本文分享Unity中的资源管理-对象池技术(3) 在上两篇文章中, 我们一起学习了普通类的两种对象池实现, 今天接着介绍Unity中GameObject(主要是预制实例化对象)的对象池. GameOb ...
Unity对象池技术
Unity对象池技术 https://blog.csdn.net/LemonXQ/article/details/77148886 顾名思义,对象池是存放对象的缓冲区.用户可以从缓冲区中放入/取出对象 ...
赵英时遥感原理分析和应用课件_细数5种停车场防砸车技术原理分析与应用
原标题:细数5种停车场防砸车技术原理分析与应用一.压力波防砸装置也叫遇阻防砸,主要是安装遇阻返回装置,当道闸杆下落过程中接触到车辆或者行人(接触力度是可以调节的),装置道闸杆底下的橡胶条受到阻力, ...
射频识别技术原理分析
射频识别技术原理分析射频识别(RFID)技术相对于传统的磁卡及IC卡技术具有非接触.阅读速度快.无磨损等特点,在最近几年里得到快速发展.为加强中国工程师对该技术的理解,本文详细介绍了RFID技术的工 ...
下拉 html 实现原理,百度搜索下拉框实现技术原理分析
问:百度搜索下拉框关键词是怎么来的,下拉框实现技术原理分析一下. 来自百度站长平台的站长回答: 对于词的研究,想必每个seoer都知道,而除了比较热门的百度相关搜索词之外,百度下拉框关键词应该也是不少 ...

Netty ObjectPool对象池技术原理分析

Netty ObjectPool对象池技术原理分析相关推荐

最新文章

热门文章