Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Restructure how agent stat data is stored #1533

Closed
Xylus opened this issue Feb 5, 2016 · 0 comments
Closed

Restructure how agent stat data is stored #1533

Xylus opened this issue Feb 5, 2016 · 0 comments
Assignees
@Xylus
Labels
enhancement
Milestone
1.6.0

Comments

@Xylus
Copy link
Contributor

Xylus commented Feb 5, 2016

The way the agent stat data is currently stored makes it inefficient to query for individual types of stat data efficiently. This makes it hard to pull the tps chart out to the main server map page and poll for near-real time update.

Improvements can be made in read efficiency and storage efficiency, which should go hand-in-hand here. Some changes that can be made:

  • Go wide instead of tall - Collapse 5 minutes worth of stat data into one row, use time delta (in seconds) as qualifiers.
  • Embed stat type into the rowkey - <salt><agentId><type><timestamp>
  • Serialize relevant data together and stuff them into a single column - would save a lot of space, and allow for per-type version management.
@Xylus Xylus self-assigned this Feb 5, 2016
@Xylus Xylus added this to the 1.5.2 milestone Feb 5, 2016
@Xylus Xylus modified the milestones: 1.5.3, 1.5.2 Mar 8, 2016
@Xylus Xylus mentioned this issue Mar 29, 2016
Closed
Xylus added a commit to Xylus/pinpoint that referenced this issue Aug 17, 2016
@Xylus
pinpoint-apm#1533 Change agent stat storage format to version 2
5b7718e
Xylus added a commit to Xylus/pinpoint that referenced this issue Aug 17, 2016
@Xylus
pinpoint-apm#1533 Add web support for agent stat storage version 2
dcdabc3
Xylus added a commit to Xylus/pinpoint that referenced this issue Aug 17, 2016
@Xylus
pinpoint-apm#1533 Change agent stat ver 2 table name
0b35c7d
@Xylus Xylus mentioned this issue Aug 18, 2016
Merged
Xylus added a commit to Xylus/pinpoint that referenced this issue Aug 19, 2016
@Xylus
pinpoint-apm#1533 Improve agent stat handler logging
880bc45 
Additional minor change - populate agentId for stat bos (not used but for completeness).
@Xylus Xylus mentioned this issue Aug 19, 2016
Merged
Xylus added a commit that referenced this issue Aug 19, 2016
@Xylus
#1533 Improve agent stat handler logging (#1996)
ad8d290 
Additional minor change - populate agentId for stat bos (not used but for completeness).
Xylus added a commit to Xylus/pinpoint that referenced this issue Aug 19, 2016
@Xylus
pinpoint-apm#1533 Fix rejected puts check when all puts were successf…
6a38b2e 
…ully queued

HbaseOperations2.asyncPut() may return null instead of an empty list if all puts are queued successfully.
@Xylus Xylus mentioned this issue Aug 19, 2016
Merged
Xylus added a commit that referenced this issue Aug 19, 2016
@Xylus
Fix rejected puts check when all puts are successfully queued (#1998)
00b9ea9 
* #1533 Fix rejected puts check when all puts were successfully queued
HbaseOperations2.asyncPut() may return null instead of an empty list if all puts are queued successfully.

* Add documentation for possible null return value
Another option would have been to modify implementations to return an empty list instead of null, but the underlying `HTableMultiplexor.put` also potentially returns null.
So to follow suit, leave the implementations alone and add documentation instead.
Xylus added a commit to Xylus/pinpoint that referenced this issue Aug 22, 2016
@Xylus
pinpoint-apm#1533 Add GC type back into JvmGc serialization
1dee6bb
@Xylus Xylus mentioned this issue Aug 22, 2016
Merged
Xylus added a commit to Xylus/pinpoint that referenced this issue Aug 24, 2016
@Xylus
pinpoint-apm#1533 Pass context when decoding agent stats
91a0785
Xylus added a commit to Xylus/pinpoint that referenced this issue Aug 24, 2016
@Xylus
pinpoint-apm#1533 Make agent stat value header encoding more flexible
0746aa1 
Agent stat value headers are no longer typed, and will be able to handle additional encoding strategies that could be added in the future.
@Xylus Xylus mentioned this issue Aug 24, 2016
Merged
Xylus added a commit that referenced this issue Aug 31, 2016
@Xylus
#1533 Fix AgentStatV2 typo
c40fc00
Xylus added a commit to Xylus/pinpoint that referenced this issue Sep 7, 2016
@Xylus
pinpoint-apm#1533 Sample average may return decimals for non-floating…
00103dc 
… point samples
@Xylus Xylus mentioned this issue Sep 7, 2016
Merged
Xylus added a commit that referenced this issue Sep 7, 2016
@Xylus
#1533 Sample average may return decimals for non-floating point sampl…
26620ab 
…es (#2064)

patch covered by integration tests in internal CI instance.
Xylus added a commit to Xylus/pinpoint that referenced this issue Sep 7, 2016
@Xylus
pinpoint-apm#1533 Add tests for DownSamplers
a47440a
Xylus added a commit that referenced this issue Sep 7, 2016
@Xylus
#1533 Add tests for DownSamplers (#2067)
5d03e81
@Xylus Xylus mentioned this issue Sep 13, 2016
Merged
@emeroad emeroad closed this as completed Oct 24, 2016
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@Xylus Xylus

Labels
enhancement
Projects
None yet
Milestone
1.6.0
Development

No branches or pull requests
2 participants
@emeroad @Xylus