Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

MySQL task resource usage storage #1928

Merged
merged 21 commits into from
May 2, 2019
Merged

MySQL task resource usage storage #1928

merged 21 commits into from
May 2, 2019

Conversation

ssalinas
Copy link
Member

@ssalinas ssalinas commented Apr 29, 2019 •
edited
Loading

Last (for now) in the 'be nice to zookeeper' series. We store cluster, slave, request, and task level resource usage, currently all in zookeeper. This does a few things to make that more scalable:

  • For request/slave we only ever need the current. Stop saving history, and make sure recent slave usages are also leader cached
  • Task usages, if in zk, need to be namespaced by request id to avoid extra large list children calls of all task ids
  • Task usages will be stored in mysql if it is configured, which will require a migration to be run to create the table
  • A zk migration has been added to clean all current usage data. This data will be rewritten the next time the poller runs and is not required for scheduling
  • task usage timestamps are now stored in millis instead of seconds
  • usage/slaves/{slaveId}/tasks/current endpoint is removed

baconmania
baconmania reviewed Apr 29, 2019
View reviewed changes
SingularityBase/src/main/java/com/hubspot/singularity/SingularityTaskUsage.java Outdated
@@ -9,7 +9,7 @@
public class SingularityTaskUsage {

private final long memoryTotalBytes;
private final double timestamp; // seconds
private final long timestamp; // seconds
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should update the comment here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, missed that

baconmania
baconmania reviewed Apr 29, 2019
View reviewed changes
SingularityService/src/main/java/com/hubspot/singularity/data/usage/TaskUsageJDBI.java Outdated

public abstract class TaskUsageJDBI {
static final String FILEDS = "requestId, taskId, memoryTotalBytes, timestamp, cpuSeconds, diskTotalBytes, cpusNrPeriods, cpusNrThrottled, cpusThrottledTimeSecs";
static final String FILED_VALUES = ":requestId, :taskId, :memoryTotalBytes, :timestamp, :cpuSeconds, :diskTotalBytes, :cpusNrPeriods, :cpusNrThrottled, :cpusThrottledTimeSecs";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: spelling

@ssalinas ssalinas added the hs_qa label Apr 30, 2019
@baconmania
Copy link
Contributor

🚢

@ssalinas ssalinas merged commit 2402934 into master May 2, 2019
@ssalinas ssalinas deleted the mysql_usage_storage branch May 2, 2019 14:46
@ssalinas ssalinas added this to the 0.23.0 milestone Jun 7, 2019
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@baconmania baconmania baconmania left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.23.0
Development

Successfully merging this pull request may close these issues.
2 participants
@ssalinas @baconmania