[RIP-9] The best practice of rocketmq #659
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vongosling
changed the title
fusiming
changed the title
fusiming
changed the title
wlliqipeng
reviewed
Jan 18, 2019
wlliqipeng
reviewed
Jan 18, 2019
|
great |
wlliqipeng
reviewed
Jan 23, 2019
wlliqipeng
reviewed
Jan 23, 2019
wlliqipeng
reviewed
Jan 23, 2019
wlliqipeng
reviewed
Jan 23, 2019
wlliqipeng
reviewed
Jan 23, 2019
wlliqipeng
reviewed
Jan 23, 2019
| | retryTimesWhenSendFailed | 2 | 如果消息发送失败，最大重试次数，该参数只对同步发送模式起作用 | | ||
| | maxMessageSize | 4MB | 客户端限制的消息大小，超过报错，同时服务端也会限制，所以需要跟服务端配合使用。 | | ||
| | transactionCheckListener | | 事务消息回查监听器，如果发送事务消息，必须设置 | | ||
| | checkThreadPoolMinSize | 1 | Broker回查Producer事务状态时，线程池大小 | |
wlliqipeng
reviewed
Jan 23, 2019
wlliqipeng
reviewed
Jan 23, 2019
| | consumeFromWhere | CONSUME_FROM_LAST_OFFSET | Consumer启动后，默认从上次消费的位置开始消费，这包含两种情况：一种是上次消费的位置未过期，则消费从上次中止的位置进行；一种是上次消费位置已经过期，则从当前队列第一条消息开始消费 | | ||
| | consumeTimestamp | 半个小时前 | 只有当consumeFromWhere值为CONSUME_FROM_TIMESTAMP时才起作用。 | | ||
| | allocateMessageQueueStrategy | AllocateMessageQueueAveragely | Rebalance算法实现策略 | | ||
| | subscription | {} | 订阅关系 | |
wlliqipeng
reviewed
Jan 23, 2019
| | subscription | {} | 订阅关系 | | ||
| | messageListener | | 消息监听器 | | ||
| | offsetStore | | 消费进度存储 | | ||
| | consumeThreadMin | 10 | 消费线程池数量 | |
wlliqipeng
reviewed
Jan 23, 2019
| | brokerSuspendMaxTimeMillis | 20000 | 长轮询，Consumer拉消息请求在Broker挂起最长时间，单位毫秒 | | ||
| | consumerTimeoutMillisWhenSuspend | 30000 | 长轮询，Consumer拉消息请求在Broker挂起超过指定时间，客户端认为超时，单位毫秒 | | ||
| | consumerPullTimeoutMillis | 10000 | 非长轮询，拉消息超时时间，单位毫秒 | | ||
| | messageModel | BROADCASTING | 消息模型，支持以下两种 1、集群消费 2、广播消费 | |
wlliqipeng
reviewed
Jan 23, 2019
wlliqipeng
reviewed
Jan 23, 2019
| | messageModel | BROADCASTING | 消息模型，支持以下两种 1、集群消费 2、广播消费 | | ||
| | messageQueueListener | | 监听队列变化 | | ||
| | offsetStore | | 消费进度存储 | | ||
| | registerTopics | [] | 注册的topic集合 | |
wlliqipeng
reviewed
Jan 23, 2019
docs/cn/best_practise.md
Outdated
| ``` | ||
| 消息发送成功或者失败要打印消息日志，务必要打印SendResult和key字段。send消息方法只要不抛异常，就代表发送成功。发送成功会有多个状态，在sendResult里定义。以下对每个状态进行说明： | ||
|
|
||
| ● SEND_OK（发送成功） |
wlliqipeng
reviewed
Jan 23, 2019
docs/cn/best_practise.md
Outdated
| ### 7.2.3选择oneway形式发送 | ||
|
|
||
| 通常消息的发送是这样一个过程： | ||
| 1) 客户端发送请求到服务器 |
wlliqipeng
reviewed
Jan 23, 2019
|
|
||
| 1) 建议消息的大小最好不要超过512K。 | ||
|
|
||
| 异步发送（AsyncSending） 默认方法send（msg）会在得到返回响应之前一直被阻塞。因此，如果用户重视性能，那么建议使用send（msg、callback），这样消息发送将以异步方式进行。 |
wlliqipeng
reviewed
Jan 23, 2019
docs/cn/best_practise.md
Outdated
| 1 提高消费并行度 | ||
|
|
||
| 绝大部分消息消费行为都属于 IO 密集型，即可能是操作数据库，或者调用 RPC，这类消费行为的消费速度在于后端数据库或者外系统的吞吐量，通过增加消费并行度，可以提高总的消费吞吐量，但是并行度增加到一定程度，反而会下降。所以，应用必须要设置合理的并行度。 如下有几种修改消费并行度的方法： | ||
| 1) 同一个 ConsumerGroup 下，通过增加 Consumer 实例数量来提高并行度（需要注意的是超过订阅队列数的 Consumer 实例无效）。可以通过加机器，或者在已有机器启动多个进程的方式。 |
wlliqipeng
reviewed
Jan 23, 2019
docs/cn/best_practise.md
Outdated
| ``` | ||
| 4 优化每条消息消费过程 | ||
| 举例如下，某条消息的消费过程如下： | ||
| 1) 根据消息从 DB 查询【数据 1】 |
wlliqipeng
reviewed
Jan 23, 2019
| ### 7.3.3 消费打印日志 | ||
|
|
||
| 如果消息量较少，建议在消费入口方法打印消息，消费耗时等，方便后续排查问题。 | ||
| ```java |
wlliqipeng
reviewed
Jan 23, 2019
|
|
||
| 7 关于消费位点 | ||
|
|
||
| 当建立一个新的消费者组时，需要决定是否需要消费已经存在于 Broker 中的历史消息CONSUME_FROM_LAST_OFFSET 将会忽略历史消息，并消费之后生成的任何消息。CONSUME_FROM_FIRST_OFFSET 将会消费每个存在于 Broker 中的信息。你也可以使用 CONSUME_FROM_TIMESTAMP 来消费在指定时间戳后产生的消息。 |
There was a problem hiding this comment.
"...于Broker中的历史消息”后面是句号
wlliqipeng
reviewed
Jan 23, 2019
|
|
||
| 推荐使用最新发布的JDK 1.8版本。通过设置相同的Xms和Xmx值来防止JVM调整堆大小以获得更好的性能。简单的JVM配置如下所示： | ||
| -server -Xms8g -Xmx8g -Xmn4g | ||
| 如果您不关心RocketMQ代理的启动时间，还有一种更好的选择，就是通过“预触摸”Java堆以确保在JVM初始化期间每个页面都将被分配。那些不关心启动时间的人可以启用它： |
wlliqipeng
reviewed
Jan 23, 2019
| -XX:G1ReservePercent=25 | ||
| -XX:InitiatingHeapOccupancyPercent=30 | ||
| ``` | ||
| 这些GC选项看起来有点激进，但事实证明它在我们的生产环境中具有良好的性能。另外不要把-XX:MaxGCPauseMillis的值设置太小，否则JVM将使用一个小的年轻代来实现这个目标，这将导致非常频繁的minor GC，所以官方建议使用rolling GC日志文件： |
wlliqipeng
reviewed
Jan 23, 2019
|
|
||
| os.sh脚本在bin文件夹中列出了许多内核参数，可以进行微小的更改然后用于生产用途。下面的参数需要注意，更多细节请参考/proc/sys/vm/*的文档。 | ||
| （<https://www.kernel.org/doc/Documentation/sysctl/vm.txt> ） | ||
| vm.extra_free_kbytes，告诉VM在后台回收（kswapd）启动的阈值与直接回收（通过分配进程）的阈值之间保留额外的可用内存.RocketMQ使用此参数来避免内存分配中的长延迟。（与具体内核版本相关） |
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
RocketMQ Developer Guide doc
Brief changelog
Add the file named best_practice.md
Verifying this change
no functional change
Follow this checklist to help us incorporate your contribution quickly and easily. Notice,
it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.[ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyleto make sure basic checks pass. Runmvn clean install -DskipITsto make sure unit-test pass. Runmvn clean test-compile failsafe:integration-testto make sure integration-test pass.