java - Cassandra 是否对压缩 OOPS 问题几乎免疫?

标签 java pointers cassandra

我正在阅读有关 Java Compressed OOPS 的内容。我发现 Java 所做的是大量转变为对所有对象引用使用压缩指针。

我能理解的是,对于大量对象,从 -XX:+UseCompressedOops-XX:-UseCompressedOops,您最终可能会使用更多内存。

假设您有 100 万个对象,在第一种情况下,您需要 4Mb 的数据用于引用,在第二种情况下,您需要 8Mb。

现在,我有一台 64Gb RAM 的计算机,并且想为 Cassandra 使用 50Gb。但现在我想知道额外的 18Gb 是否真的有用,还是会被许多无用的指针数据填满。

所以我的问题是:

Cassandra 是否管理大量小对象,还是只管理几个大对象? (即,它是否为每个单元、每一行、每个 sstable 分配一个单独的缓冲区?)

在第一种情况下,当 Cassandra 缓存超过 32Gb 与 50Gb 相比时,我们可能不会获得太多 yield 。

最佳答案

从堆的角度来看,这是我的一个集群节点的样子:

# /usr/lib/jvm/jdk1.8.0_121/bin/jmap -histo -F 2480
Attaching to process ID 2480, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.121-b13
Iterating over heap. This may take a while...
Object Histogram:

num       #instances    #bytes  Class description
--------------------------------------------------------------------------
1:              1300610 1008416856      byte[]
2:              8451955 405693840       java.nio.HeapByteBuffer
3:              2349359 75179488        org.apache.cassandra.db.BufferCell
4:              2723073 73455464        java.nio.ByteBuffer[]
5:              1804559 57745888        org.apache.cassandra.db.composites.CompoundSparseCellName
6:              510867  46169960        java.lang.Object[]
7:              989235  23741640        java.util.concurrent.ConcurrentSkipListMap$Node
8:              25068   20455288        double[]
9:              851046  20425104        org.apache.cassandra.db.composites.CompoundDenseCellName
10:             580063  18562016        com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node
11:             580056  18561792        com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8$Node
12:             42317   15771032        long[]
13:             600999  14423976        com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$WeightedValue
14:             590310  14167440        org.apache.cassandra.cache.KeyCacheKey
15:             554649  13311576        org.apache.cassandra.db.RowIndexEntry
16:             35239   12545512        int[]
17:             434971  10439304        java.lang.Long
18:             143327  9172928 java.nio.DirectByteBufferR
19:             354204  8500896 java.lang.String
20:             98616   8241104 char[]
21:             37512   8102592 com.codahale.metrics.WeightedSnapshot$WeightedSample[]
22:             181096  7243840 org.apache.cassandra.db.BufferExpiringCell
23:             284486  6827664 org.apache.cassandra.db.AtomicBTreeColumns$Holder
24:             264728  6353472 org.apache.cassandra.cql3.ColumnIdentifier
25:             264405  6345720 java.util.concurrent.ConcurrentSkipListMap$Index
26:             122067  4882680 org.apache.cassandra.io.sstable.IndexHelper$IndexInfo
27:             1       4194320 com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8$Node[]
28:             99838   2396112 org.apache.cassandra.dht.Murmur3Partitioner$LongToken
29:             89034   2136816 org.apache.cassandra.db.BufferDecoratedKey
30:             88888   2133312 org.apache.cassandra.db.AtomicBTreeColumns
31:             64001   2048032 org.apache.cassandra.db.composites.CompoundSparseCellName$WithCollection
32:             58297   1399128 java.util.ArrayList
33:             56381   1353144 java.util.EnumMap$EntryIterator$Entry
34:             26794   1286112 java.util.HashMap
35:             37450   1198400 java.util.HashMap$Node
36:             47639   1143336 java.lang.Double
37:             35632   1140224 com.codahale.metrics.WeightedSnapshot$WeightedSample
38:             24311   972440  java.util.TreeMap$Entry
39:             18219   874512  org.apache.cassandra.io.sstable.format.big.SSTableNamesIterator
40:             35651   855624  org.apache.cassandra.db.composites.BoundedComposite
41:             35610   854640  org.apache.cassandra.db.composites.CompoundComposite
42:             26165   837280  java.util.TreeMap$KeyIterator
43:             1248    818688  io.netty.util.internal.shaded.org.jctools.queues.MpscArrayQueue
44:             25419   813408  org.apache.cassandra.db.RowIndexEntry$IndexedEntry
45:             50021   800336  java.lang.Integer
46:             6585    736576  java.lang.Class
47:             4182    736032  org.apache.cassandra.net.MessagingService$Verb[]
48:             7718    652400  java.util.HashMap$Node[]
49:             19391   620512  java.util.concurrent.ConcurrentHashMap$Node
50:             9349    600624  java.util.Hashtable$Entry[]
51:             12504   600192  java.util.TimSort
52:             18661   597152  java.util.Vector
53:             4100    557600  com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$PaddedAtomicReference
54:             4953    554736  java.net.SocksSocketImpl
55:             13215   528656  java.util.concurrent.ConcurrentSkipListMap$Index[]
56:             15888   508416  java.util.concurrent.ConcurrentSkipListMap$HeadIndex
57:             19379   465096  org.apache.cassandra.db.ArrayBackedSortedColumns$4
58:             9339    448272  java.util.Hashtable
59:             4520    419824  short[]
60:             25970   415520  java.lang.Object
61:             16148   387552  javax.management.ObjectName$Property
62:             9358    374320  java.security.ProtectionDomain
63:             12504   300096  com.codahale.metrics.WeightedSnapshot
64:             9356    299392  java.security.CodeSource
65:             12367   296808  java.util.concurrent.ConcurrentLinkedDeque$Node
66:             5891    282768  org.apache.cassandra.net.MessageOut
67:             5676    272448  java.util.TreeMap
68:             8542    271256  javax.management.ObjectName$Property[]
69:             8034    257088  java.util.EnumMap$EntryIterator
70:             7077    226464  java.net.InetAddress$InetAddressHolder
71:             7072    226304  java.util.ArrayList$Itr
72:             5644    225760  org.apache.cassandra.db.PreHashedDecoratedKey
73:             9288    222912  java.util.concurrent.ConcurrentLinkedQueue$Node
74:             3085    222120  com.sun.jmx.remote.util.OrderClassLoaders
75:             3085    222120  javax.management.remote.rmi.RMIConnectionImpl$CombinedClassLoader
76:             3085    222120  javax.management.remote.rmi.RMIConnectionImpl$CombinedClassLoader$ClassLoaderWrapper
77:             6730    215360  java.util.concurrent.locks.AbstractQueuedSynchronizer$Node
78:             8413    201912  org.apache.cassandra.db.DeletionInfo
79:             12504   200064  com.codahale.metrics.WeightedSnapshot$1
80:             4096    196608  java.util.concurrent.ConcurrentHashMap$TreeNode
81:             250     196544  java.util.concurrent.ConcurrentHashMap$Node[]
82:             4881    195240  org.apache.cassandra.db.ArrayBackedSortedColumns
83:             4800    192000  java.util.ArrayList$SubList
84:             150     191840  io.netty.buffer.PoolSubpage[]
85:             3952    189696  org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask
86:             7635    183240  org.apache.thrift.protocol.TField
87:             3716    178368  org.apache.cassandra.db.SliceByNamesReadCommand
88:             2778    177792  io.netty.buffer.PoolSubpage
89:             1979    174152  java.lang.reflect.Method
90:             4271    170840  javax.management.ObjectName
91:             3522    169056  org.apache.cassandra.io.sstable.Descriptor
92:             2340    168480  org.apache.thrift.protocol.TBinaryProtocol
93:             2990    167240  org.apache.cassandra.db.Cell[]
94:             5076    162432  org.apache.cassandra.db.ArrayBackedSortedColumns$ForwardsCellIterator
95:             425     159800  java.lang.Thread
96:             3988    159520  com.google.common.collect.SingletonImmutableBiMap
97:             4949    158368  java.net.Socket
98:             6548    157152  java.net.Inet4Address
99:             4870    155840  sun.misc.FDBigInteger
100:            9730    155680  java.util.HashSet
[...]
2865:           1       16      org.apache.cassandra.service.PendingRangeCalculatorService$1
2866:           1       16      sun.reflect.generics.tree.VoidDescriptor
2867:           1       16      org.apache.cassandra.thrift.Cassandra$Processor$insert
Total :         25759325        -Xms7967M -Xmx7967M -Xmn800M
Heap traversal took 1130.769 seconds.

堆上有 25.759.325 个实例,该节点以 -Xms7967M -Xmx7967M -Xmn800M 启动 - 因此这里有相当多的实例。

有关调整资源,另请参阅:https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/operations/opsTuneJVM.html

在我的一项测试中,我最终在一台主机上运行 4 个 Cassandra 实例,每个实例有 8GB 堆(在一台具有 64GB RAM 的机器上,为操作系统页面缓存留出 32GB,确保每个主机具有机架感知能力)当一台主机宕机时丢失多个副本)。

关于java - Cassandra 是否对压缩 OOPS 问题几乎免疫?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/54033180/

相关文章:

java - 工具栏中的SearchView不过滤ListFragment

JavaFX 更改不同类的 Pane 颜色

c - 这个代码片段具体做了什么?

java - 需要添加允许访问 Cassandra 过滤

docker - COPY在cassandra中生成的导出csv文件的默认目录在哪里?

java - 如何连接到在 Docker 中运行的远程 Cassandra 实例以使用 Java 检索数据?

java - mule/java - 从 java 类读取 jar 依赖项内部的属性文件

java - NoSuchMethodError - org.hibernate.cfg.Environment

c++ - 将迭代器转换为指针?

c - 打印内存位置时 Printf 产生警告但仍在 xcode 中编译和运行