2019独角兽企业重金招聘Python工程师标准>>>

Java was initially designed as a safe, managed environment. Nevertheless, Java HotSpot VM contains a “backdoor” that provides a number of low-level operations to manipulate memory and threads directly. This backdoor – sun.misc.Unsafe – is widely used by JDK itself in the packages like java.nio or java.util.concurrent. It is hard to imagine a Java developer who uses this backdoor in any regular development because this API is extremely dangerous, non portable, and volatile. Nevertheless, Unsafe provides an easy way to look into HotSpot JVM internals and do some tricks. Sometimes it is simply funny, sometimes it can be used to study VM internals without C++ code debugging, sometimes it can be leveraged for profiling and development tools.

Obtaining Unsafe

The sun.misc.Unsafe class is so unsafe that JDK developers added special checks to restrict access to it. Its constructor is private and caller of the factory method getUnsafe() should be loaded by Bootloader (i.e. caller should also be a part of JDK):

1
2
3
4
5
6
7
8
9
10
11
12
13
publicfinalclassUnsafe {
...
privateUnsafe() {}
privatestaticfinalUnsafe theUnsafe =newUnsafe();
...
publicstaticUnsafe getUnsafe() {
Class cc = sun.reflect.Reflection.getCallerClass(2);
if(cc.getClassLoader() !=null)
thrownewSecurityException("Unsafe");
returntheUnsafe;
}
...
}

Fortunately there is theUnsafe field that can be used to retrieve Unsafe instance. We can easily write a helper method to do this via reflection:

1
2
3
4
5
6
7
publicstaticUnsafe getUnsafe() {
try{
Field f = Unsafe.class.getDeclaredField("theUnsafe");
f.setAccessible(true);
return(Unsafe)f.get(null);
}catch(Exception e) {/* ... */}
}

In the next sections we will study several tricks that become possible due to the following methods of Unsafe:

  • long getAddress(long address) and void putAddress(long address, long x) that allows to read and write dwords directly from memory.
  • int getInt(Object o, long offset) , void putInt(Object o, long offset, int x), and other similar methods that allows to read and write data directly from C structure that represents Java object.
  • long allocateMemory(long bytes) which can be considered as a wrapper for C’s malloc().

sizeof() Function

The first trick we will do is C-like sizeof() function, i.e. function that returns shallow object size in bytes. Inspecting JVM sources of JDK6 and JDK7, in particular src/share/vm/oops/oop.hpp and src/share/vm/oops/klass.hpp, and reading comments in the code, we can notice that size of class instance is stored in _layout_helper which is the fourth field in C structure that represents Java class. Similarly, /src/share/vm/oops/oop.hpp shows that each instance (i.e. object) stores pointer to a class structure in its second field. For 32-bit JVM this means that we can first take class structure address as 4-8 bytes in the object structure and next shift by 3×4=12 bytes inside class structure to capture_layout_helper field which is instance size in bytes. These structures are shown in the picture below:

As so, we can implement sizeof() as follows:

1
2
3
4
5
6
7
8
9
publicstaticlongsizeOf(Object object) {
Unsafe unsafe = getUnsafe();
returnunsafe.getAddress( normalize( unsafe.getInt(object, 4L) ) + 12L );
}
publicstaticlongnormalize(intvalue) {
if(value >=0)returnvalue;
return(~0L >>>32) & value;
}

We need to use normalize() function because addresses between 2^31 and 2^32 will be automatically converted to negative integers, i.e. stored in complement form. Let’s test it on 32-bit JVM (JDK 6 or 7):

1
2
3
4
5
// sizeOf(new MyStructure()) gives the following results:
classMyStructure { }// 8: 4 (start marker) + 4 (pointer to class)
classMyStructure {intx; }// 16: 4 (start marker) + 4 (pointer to class) + 4 (int) + 4 stuff bytes to align structure to 64-bit blocks
classMyStructure {intx;inty; }// 16: 4 (start marker) + 4 (pointer to class) + 2*4

This function will not work for array objects, because _layout_helper field has another meaning in that case. Although it is still possible to generalize sizeOf() to support arrays.

Direct Memory Management

Unsafe allows to allocate and deallocate memory explicitly via allocateMemory and freeMemory methods. Allocated memory is not under GC control and not limited by maximum JVM heap size. In general, such functionality is safely available via NIO’s off-heap bufferes. But the interesting thing is that it is possible to map standard Java reference to off-heap memory:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
MyStructure structure =newMyStructure();// create a test object
structure.x =777;
longsize = sizeOf(structure);
longoffheapPointer = getUnsafe().allocateMemory(size);
getUnsafe().copyMemory(
structure,     // source object
0,             // source offset is zero - copy an entire object
null,          // destination is specified by absolute address, so destination object is null
offheapPointer,// destination address
size
);// test object was copied to off-heap
Pointer p =newPointer();// Pointer is just a handler that stores address of some object
longpointerOffset = getUnsafe().objectFieldOffset(Pointer.class.getDeclaredField("pointer"));
getUnsafe().putLong(p, pointerOffset, offheapPointer);// set pointer to off-heap copy of the test object
structure.x =222;// rewrite x value in the original object
System.out.println(  ((MyStructure)p.pointer).x  );// prints 777
....
classPointer {
Object pointer;
}

So, it is virtually possible to manually allocate and deallocate real objects, not only byte buffers. Of course, it’s a big question what may happen with GC after such cheats.

Inheritance from Final Class and void*

Imagine the situation when one has a method that takes a string as an argument, but it is necessary to pass some extra payload. There are at least two standard ways to do it in Java: put payload to thread local or use static field. With Unsafe another two possibilities appears: pass payload address as a string and inherit payload class from String class. The first approach is pretty close to what we see in the previous section – one just need obtain payload address using Pointer and create a new Pointer to payload inside the called method. In other words, any argument that can carrier an address can be used as analog of void* in C. In order to explore the second approach we start with the following code which is compilable, but obviously produces ClassCastException in run time:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Carrier carrier =newCarrier();
carrier.secret =777;
String message = (String)(Object)carrier;// ClassCastException
handler( message );
...
voidhandler(String message) {
System.out.println( ((Carrier)(Object)message).secret );
}
...
classCarrier {
intsecret;
}

To make it work, one need to modify Carrier class to simulate inheritance from String. A list of superclasses is stored in Carrier class structure starting from position 28, as it shown in the figure. Pointer to object goes first and pointer to Carrier itself goes after it (at position 32) since Carrier is inherited from Object directly. In principle, it is enough to add the following code before the line that casts Carrier to String:

1
2
3
longcarrierClassAddress = normalize( unsafe.getInt(carrier, 4L) );
longstringClassAddress = normalize( unsafe.getInt("", 4L) );
unsafe.putAddress(carrierClassAddress +32, stringClassAddress);// insert pointer to String class to the list of Carrier's superclasses

Now cast works fine. Nevertheless, this transformation is not correct and violates VM contracts. More careful approach should include more steps:

  1. Position 32 in Carrier class actually contains a pointer to Carrier class itself, so this pointer should be shifted to position 36, not simply overwritten by the pointer to the String class.
  2. Since Carrier is now inherited from String, final markers in String class should be removed.

Conclusion

sun.misc.Unsafe provides almost unlimited capabilities for exploring and modification of VM’s runtime data structures. Despite the fact that these capabilities are almost inapplicable in Java development itself, Unsafe is a great tool for anyone who want to study HotSpot VM without C++ code debugging or need to create ad hoc profiling instruments.

转载于:https://my.oschina.net/u/138995/blog/213506

Tricks with Direct Memory Access in Java相关推荐

  1. Remote Direct Memory Access (RDMA)

    前言 本博文主要是简单介绍RDMA的概念和与它相关的技术.实现RDMA需要许多其他技术的支持,包括硬件和软件.目前RDMA有多种实现方式,比如RoCE1.INFINIBAND2,不同的实现方式所考虑的 ...

  2. 直接内存访问 (Direct Memory Access, DMA)

    Table of Contents 直接内存访问 (Direct Memory Access, DMA) DMA 模型 设备 DMA 的类型 总线主控器 DMA 第三方 DMA 第一方 DMA 主机平 ...

  3. Xilinx AXI Central Direct Memory Access (CDMA)手册笔记

    官方手册地址:https://china.xilinx.com/support/documentation/ip_documentation/axi_cdma/v4_1/pg034-axi-cdma. ...

  4. DAM(Direct Memory Access)

    cpu 和 IO 之间的协作关系 占用-->中断-->异步(IOP专门做IO操作) 以往的io操作都需要经过cpu之手,在以前,cpu要一直轮询io系统,io操作是否已经结束,在这期间,c ...

  5. RDMA,remote direct memory access

    在电脑运算领域,远程直接内存访问(英语:remote direct memory access,RDMA)是一种直接内存访问技术,它将数据直接从一台计算机的内存传输到另一台计算机,无需双方操作系统的介 ...

  6. DMA(direct memory access)直接内存访问

    2019独角兽企业重金招聘Python工程师标准>>> DMA 实际上是盗用了总线时间来传输数据,而且由于是硬件处理,所以大大加速了数据复制速度! 1. 基本概念 辅存狭义上是平时讲 ...

  7. 直接存储器存取(Direct Memory Access,DMA)详细讲解

    一.理论理解部分. 1.直接存储器存取(DMA)用来提供在外设和存储器之间或者存储器和存储器之间的高速数据传输. 2.无须CPU干预,数据可以通过DMA快速移动,这就节省了CPU的资源来做其他操作. ...

  8. DMA(direct memory access)控制方式

    1.DMA控制方式的引入        中断方式是以字节(字)为单位进行I/O的, 每完成一个字节(字) CPU要处理一次中断, 这种方法用于块设备效率极低, 为了进一步减少CPU对I/O的干预, 提 ...

  9. Java用JNA调用dll : Invalid memory access

    问题描述 java通过JNA调用C/C++ dll时,报Invalid memory access 问题原因 经过分析原因是数据类型不匹配问题 int &a 和 a 的区别 C语言中的a是一个 ...

最新文章

  1. TinyXML2 的使用
  2. linux+python+djiango+mysql编译安装学习笔记
  3. linux下su和su - 的区别
  4. 安卓初学者必看实例,(手机GPS简单编程访问)
  5. modbus-rtu qt4-serialport1------ xp as host
  6. idea资源包下创建资源包_资源包技巧和最佳实践
  7. yxcms安装环境php,Windows7下PHP开发环境安装配置图文方法
  8. 信息学奥赛一本通(1169:大整数减法)
  9. python深入和扩展_用Python来扩展Postgresql(一)
  10. zabbix的boot.log占满根目录不能ssh连接No space left on device
  11. 5. Linux 设备文件名
  12. flash 模拟eeprom
  13. 企业微信h5开发(即JS-SDK),一不小心,就会掉进坑,进入死胡同
  14. SQLAlchemy 字段、要点
  15. SAP中销售订单流程及常用事务 Get the picture
  16. 史上最全的Java面试题集锦在这里,带你碾压面试官!
  17. TienLen游戏模型、算法,类似斗地主游戏算法
  18. 【总结】1026- 一文读懂 base64
  19. 舵机 - 什么是舵机
  20. TextCNN文本分类实现(主要是CNN模型的使用)

热门文章

  1. xgboost分类器直接调用验证集的评估结果
  2. property、staticmethod、classmethod与__str__的用法
  3. adam优化器再理解
  4. mysql跨库查询 索引_MySQL中跨库查询怎么搞?
  5. excel可视化图表插件_Axure 教程:利用图表前端插件实现高级可视化图表
  6. linux内核关闭触摸屏校准,linux内核usb触摸屏驱动bug调试- selected device is not a touchscreen I understand...
  7. 笔记-项目管理ITTO-高项/PMP第五版-全
  8. 笔记-信息系统安全管理-安全审计-作用
  9. Leaflet中使用MovingMarker插件实现标记移动(轨迹回放效果)
  10. Electron中与Vue集成流程