-- Logs begin at Mon 2018-08-20 18:52:42 MSK, end at Thu 2018-08-23 15:45:45 MSK. -- Aug 23 15:25:33 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: ==> Starting Nomad agent... Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: ==> Nomad agent configuration: Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: Client: false Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: Log Level: INFO Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: Region: amrv01 (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: Server: true Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: Version: 0.8.4 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.616714 [INFO] nomad: starting 4 scheduling worker(s) for [batch system service _core] Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: Attempting re-join to previously known node: ip-172-29-4-194.atf01: 172.29.4.194:4648 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.616846 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795012 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795033 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795046 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795058 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795412 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795474 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795491 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 2018/08/23 15:25:33.795503 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795517 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795528 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795540 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795582 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795615 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795713 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795731 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795744 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.795755 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33 [INFO] serf: Re-joined to previously known node: ip-172-29-4-194.atf01: 172.29.4.194:4648 Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.796000 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.796018 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:25:33 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:33.796035 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:25:35 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:35 [WARN] memberlist: Refuting a suspect message (from: ip-172-24-20-236.amrv01) Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41.370827 [INFO] server.nomad: successfully contacted 20 Nomad Servers Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:25:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:41 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:25:42 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:42 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:25:42 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:25:42 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.457172 [ERR] worker: failed to update evaluation : rpc error: leadership lost while committing log Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.457189 [ERR] worker: failed to update evaluation : rpc error: leadership lost while committing log Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.457186 [ERR] worker: failed to update evaluation : rpc error: leadership lost while committing log Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.458214 [ERR] worker: failed to nack evaluation '83f47d59-5cb6-71e3-a07d-8bc26f84b4c7': rpc error: No cluster leader Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.458219 [ERR] worker: failed to nack evaluation '9da05b9e-8fc6-b175-6e96-aa3a1e4672af': rpc error: No cluster leader Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.458586 [ERR] worker: failed to nack evaluation 'e3c855b7-5e86-c87b-47be-fe40dc451a86': rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.459096 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.459205 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.459479 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.472428 [ERR] worker: failed to update evaluation : rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.473339 [ERR] worker: failed to nack evaluation '2cfeba5f-d617-9f69-c026-96640c2d5a1c': rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.474259 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.480214 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.480243 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.480474 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.495404 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.579859 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.579862 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.579879 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.579909 [ERR] worker: failed to dequeue evaluation: rpc error: EOF Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.901148 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.901178 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.901152 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:26:37 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:37.901189 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:26:39 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:39.182265 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:39 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:39.182289 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:26:39 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:39.182277 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:26:39 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:39.182309 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:26:40 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:40 [WARN] raft: Heartbeat timeout from "172.24.0.121:4647" reached, starting election Aug 23 15:26:40 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:40 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 12 Aug 23 15:26:40 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:40 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [WARN] raft: Election timeout reached, restarting election Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 13 Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [INFO] raft: Election won. Tally: 2 Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41.949005 [INFO] nomad: cluster leadership acquired Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [WARN] raft: AppendEntries to {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} rejected, sending older logs (next: 1597318) Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:41 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:41 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: 2018/08/23 15:26:42 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: fatal error: concurrent map read and map write Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 440 [running]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc4204bd520 sp=0xc4204bd500 pc=0x42d161 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: runtime.mapaccess2(0x1844be0, 0xc42035a450, 0xc4204bd5e0, 0xc420df61a0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc4204bd568 sp=0xc4204bd520 pc=0x409c75 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc42037e240, 0x1d6dd40, 0xc420df61a0, 0x0, 0xed3109f02) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:156 +0xdc fp=0xc4204bd650 sp=0xc4204bd568 pc=0xf215ac Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc42048c000, 0x1d71840, 0xc421fcd700) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc4204bd7c8 sp=0xc4204bd650 pc=0xf56bb7 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: runtime.goexit() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc4204bd7d0 sp=0xc4204bd7c8 pc=0x45c711 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 1 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420358070, 0xc42049f900) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420358070, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc420481400, 0xc420481400, 0xc4200d63a0, 0x37) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204ddf68) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: main.main() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 5 [syscall, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: os/signal.signal_recv(0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: os/signal.loop() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by os/signal.init.0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 442 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420266480) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 385 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421322840, 0xc42285bd10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421322840, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc4202163d0, 0xc42208efa0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc4202163d0, 0xc42208efa0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 120 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc420220180, 0x1d71840, 0xc42001e040) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 7 [select, 1 minutes, locked to thread]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: runtime.selectgo(0xc4204bb750, 0xc420090240) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: runtime.ensureSigM.func1() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: runtime.goexit() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 32 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc4204041c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 33 [sleep]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: time.Sleep(0x3b9aca00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420456bb0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 50 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc4204ee480) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 51 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4201f6300) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 420 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421322840) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 53 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4204a09c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc42035aed0, 0xc42045e560) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 418 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e6767b0, 0x72, 0xc422d69ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420560b18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420560b18, 0xc4224e6000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc420560b00, 0xc4224e6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc420560b00, 0xc4224e6000, 0x1000, 0x1000, 0xc422d69dd8, 0x0, 0xc422d69e08) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc4202163d0, 0xc4224e6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc422384180, 0xc42276ce10, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc422384180, 0xc42276ce10, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc422384180, 0xc42276ce10, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421322840, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421322840) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 364 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676a20, 0x72, 0xc4204d2ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc4219e2998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc4219e2998, 0xc422e89000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc4219e2980, 0xc422e89000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc4219e2980, 0xc422e89000, 0x1000, 0x1000, 0xc4204d2dd8, 0x0, 0xc4204d2e08) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420276798, 0xc422e89000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc421d62c00, 0xc42276ca00, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc421d62c00, 0xc42276ca00, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc421d62c00, 0xc42276ca00, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421e52790, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421e52790) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 232 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421572370) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 249 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420457b80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 43 [IO wait, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676f00, 0x72, 0xc4200609a8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc42022a398, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc42022a398, 0xc42047c000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc42022a380, 0xc42047c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc42022a380, 0xc42047c000, 0x1000, 0x1000, 0xc420060e58, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420276148, 0xc42047c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*persistConn).Read(0xc420436240, 0xc42047c000, 0x1000, 0x1000, 0xc42001c000, 0xc420436240, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc420238600) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Peek(0xc420238600, 0x1, 0x0, 0x0, 0x0, 0xc420dd8f60, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*persistConn).readLoop(0xc420436240) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by net/http.(*Transport).dialConn Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 243 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4201f6600, 0xc421093208, 0x20, 0x20, 0xc420cdd180, 0xc421374300) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421374510, 0x1d718c0, 0xc4201f6600, 0x0, 0xc4201f6600) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42048f680, 0xc421093440, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Job).List(0xc4202261c8, 0xc4201f6540, 0xc421374300, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/job_endpoint.go:925 +0x2ea Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: reflect.Value.call(0xc4201f7680, 0xc4202266d8, 0x13, 0x1bda9a0, 0x4, 0xc421093898, 0x3, 0x3, 0x192b7e0, 0x19, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: reflect.Value.Call(0xc4201f7680, 0xc4202266d8, 0x13, 0xc421252898, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*service).call(0xc420220340, 0xc42020f450, 0xc420896160, 0x0, 0xc42046f880, 0xc42076a0a0, 0x1974e60, 0xc4201f6540, 0x16, 0x169dd20, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc42020f450, 0x1d71b80, 0xc420dda080, 0xc4201fe000, 0xa) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc42048f680, 0x1be08be, 0x8, 0x1974e60, 0xc4201f64e0, 0x169dd20, 0xc4213742d0, 0xc4201f6401, 0xc4213742d0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc420378070, 0x1be08be, 0x8, 0x1974e60, 0xc4201f64e0, 0x169dd20, 0xc4213742d0, 0x0, 0xc4201fe000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).jobListRequest(0xc420e2c340, 0x1d6df80, 0xc4201f60c0, 0xc420c06400, 0x44774f, 0xc4201eb080, 0xc4202798f0, 0x1d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:33 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest(0xc420e2c340, 0x1d6df80, 0xc4201f60c0, 0xc420c06400, 0xc420279920, 0xc421252b58, 0x38, 0xc421252b50) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:18 +0x1a6 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest-fm(0x1d6df80, 0xc4201f60c0, 0xc420c06400, 0xc420c06400, 0xc42070e1e0, 0x1d, 0xbed7c7d1884a1d0f) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:141 +0x48 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc4201f60c0, 0xc420c06400) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.HandlerFunc.ServeHTTP(0xc420749520, 0x1d6df80, 0xc4201f60c0, 0xc420c06400) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*ServeMux).ServeHTTP(0xc420fce120, 0x1d6df80, 0xc4201f60c0, 0xc420c06400) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc420190380, 0xc420c06400) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.HandlerFunc.ServeHTTP(0xc420fce2a0, 0x1d6fec0, 0xc420190380, 0xc420c06400) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.serverHandler.ServeHTTP(0xc4201ba5b0, 0x1d6fec0, 0xc420190380, 0xc420c06400) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*conn).serve(0xc4214e86e0, 0x1d71840, 0xc42121c000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by net/http.(*Server).Serve Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 44 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*persistConn).writeLoop(0xc420436240) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by net/http.(*Transport).dialConn Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 192 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420f9f970, 0xc4211ded10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420f9f970, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420216c28, 0xc420e0a7d0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420216c28, 0xc420e0a7d0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 54 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc42020f6d0, 0x1c636a0, 0xc420456e70, 0x1d71840, 0xc4200d3ac0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc420456e70) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 15 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4202a6840) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a6840, 0xc421378bf0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 441 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420266480) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 245 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4215736b0, 0xc421251d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4215736b0, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420216bc0, 0xc42096a690) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420216bc0, 0xc42096a690) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 419 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421322840) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 363 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421e52790, 0xc4211dfd10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421e52790, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420276798, 0xc42208ecd0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420276798, 0xc42208ecd0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 16 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4202a6840) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a6840, 0xc421378c00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 274 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420f9f970) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 14 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4202a6840) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4202a6840) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4202a6840) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:26:42 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a6840, 0xc421378be0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 193 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e6762d0, 0x72, 0xc4201b4ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc421083d18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc421083d18, 0xc421978000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc421083d00, 0xc421978000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc421083d00, 0xc421978000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420216c28, 0xc421978000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc42256e900, 0xc42242d8f0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc42256e900, 0xc42242d8f0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc42256e900, 0xc42242d8f0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420f9f970, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420f9f970) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 130 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420ed9590) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 131 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4201ba0d0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 132 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676d60, 0x72, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc42070a918, 0x72, 0xc420082000, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc42070a918, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Accept(0xc42070a900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).accept(0xc42070a900, 0xc4202175c8, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*TCPListener).accept(0xc420227430, 0x458e40, 0xc42281ef90, 0xc42281ef98) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*TCPListener).AcceptTCP(0xc420227430, 0x1c63658, 0xc420c663c0, 0xc4202175c8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420c663c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 133 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676c90, 0x72, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc42070ac18, 0x72, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc42070ac18, 0xc422c04000, 0x10000, 0x10000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).ReadFrom(0xc42070ac00, 0xc422c04000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).readFrom(0xc42070ac00, 0xc422c04000, 0x10000, 0x10000, 0x96, 0x96, 0x1d62b40, 0x412539, 0xc4203b1e40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*UDPConn).readFrom(0xc420227438, 0xc422c04000, 0x10000, 0x10000, 0x101, 0xc4203b1e18, 0xc4203b1e90, 0x443cf7) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*UDPConn).ReadFrom(0xc420227438, 0xc422c04000, 0x10000, 0x10000, 0x10000, 0x10000, 0xa200, 0xf3435a080, 0x2b96de0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420c663c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 134 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420c663c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 135 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420c663c0, 0x12a05f200, 0xc4208d7b60, 0xc420f8d620, 0xc421379110) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 136 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420c663c0, 0xc420f8d620) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:143 +0x1f3 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 137 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420c663c0, 0x1dcd6500, 0xc4208d7bc0, 0xc420f8d620, 0xc421379120) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 138 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4201ab800) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 139 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4201ab800) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 140 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4201ab800, 0x1bdd09a, 0x6, 0xc420ed96e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 141 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4201ab800, 0x1bdba6c, 0x5, 0xc420ed9710) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 142 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4201ab800, 0x1bdbc6a, 0x5, 0xc420ed9740) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 189 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e6766e0, 0x72, 0xc4204b6658) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420560918, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420560918, 0xc420fff300, 0x1, 0x1) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc420560900, 0xc420fff391, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc420560900, 0xc420fff391, 0x1, 0x1, 0xc4204b67ac, 0xc4204b67a8, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420216178, 0xc420fff391, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*connReader).backgroundRead(0xc420fff380) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 144 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4208d7c20, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4208d7c20, 0x1dcd6500, 0x24, 0xc422b80a20, 0x24, 0xc4229d1400, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4208d7c20) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 145 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4208d7c80, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4208d7c80, 0x1dcd6500, 0x24, 0xc4229d84b0, 0x24, 0xc420cc5200, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4208d7c80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 146 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4208d7ce0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4208d7ce0, 0x1dcd6500, 0x24, 0xc422b30660, 0x24, 0xc4229d1700, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4208d7ce0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 147 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4208d7d40, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4208d7d40, 0x1dcd6500, 0x24, 0xc422b30840, 0x24, 0xc421dd2800, 0xc420e86300) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4208d7d40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 148 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420fce060, 0xc42048f680) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 149 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc42048f680) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 150 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc42048f680) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 151 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676e30, 0x72, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420135418, 0x72, 0xc420082700, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420135418, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Accept(0xc420135400, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).accept(0xc420135400, 0x1, 0x1, 0xc4204d1f20) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*TCPListener).accept(0xc420226e10, 0x406b4a, 0xc420f569c0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*TCPListener).Accept(0xc420226e10, 0xc420f569c0, 0x0, 0x3f800000, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 152 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc42048c000, 0x3b9aca00, 0xc4202fe4e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 153 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc42020e410, 0x3b9aca00, 0xc4202fe4e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 154 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420266480, 0x3b9aca00, 0xc4202fe4e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 155 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4204a09c0, 0x3b9aca00, 0xc4202fe4e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 156 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc42048f680) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 157 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676af0, 0x72, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420822098, 0x72, 0xc420082200, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420822098, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Accept(0xc420822080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).accept(0xc420822080, 0x402037, 0xc421a7ebc0, 0x1805ec0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*TCPListener).accept(0xc4202276d0, 0x458e40, 0xc4211ebe90, 0xc4211ebe98) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*TCPListener).Accept(0xc4202276d0, 0x1c65af8, 0xc421a7eb40, 0x1d71900, 0xc420fce360) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*Server).Serve(0xc4201ba5b0, 0x1d6fc00, 0xc4202276d0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.Serve(0x1d6fc00, 0xc4202276d0, 0x1d5df00, 0xc420fce2a0, 0x0, 0x7fc7f1efc2d9) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420e2c340, 0x1d6fc00, 0xc4202276d0, 0xc4213795b0, 0xc420fce120) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 158 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420822100, 0xc4213795c0, 0xc420eec9c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 229 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204ee600, 0xc421d2b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc4227e4de0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc4227e4de0, 0xc421483a70, 0x4120a6, 0x7fc80e61dcd8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421de7b30, 0x3c) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4219f4320) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4227e4e40, 0x16e4b60, 0xc4219f4380) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4227e4e40, 0x16e4b60, 0xc4219f4380, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42019cb80, 0x16e4b60, 0xc4219f4380, 0xc4219f4380, 0xc4215f3ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42019cb80, 0xc4219f4380, 0x2d, 0xc422d1aee0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4215f3cc0, 0x1d72340, 0xc42019cb80, 0x60, 0x2, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4215f3cc0, 0x1d72340, 0xc42019cb80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4215f3cc0, 0x1d72340, 0xc42019cb80, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc4204ee600, 0xc4215f3cc0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 230 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676470, 0x72, 0xc421256ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420135b98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420135b98, 0xc42199d000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc420135b80, 0xc42199d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc420135b80, 0xc42199d000, 0x1000, 0x1000, 0xc421256dd8, 0x0, 0xc421256e08) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420216998, 0xc42199d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc421327ec0, 0xc4203d8000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc421327ec0, 0xc4203d8000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc421327ec0, 0xc4203d8000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421572370, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421572370) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 416 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421e529a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 163 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676950, 0x72, 0xc4211ec9a8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420822798, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420822798, 0xc420d80000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc420822780, 0xc420d80000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc420822780, 0xc420d80000, 0x1000, 0x1000, 0x458600, 0xc420201080, 0x4) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420227708, 0xc420d80000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*persistConn).Read(0xc420a46360, 0xc420d80000, 0x1000, 0x1000, 0xc4211ecb98, 0x405675, 0xc420a054a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc420d62720) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Peek(0xc420d62720, 0x1, 0x0, 0x0, 0x1, 0xc42282df80, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*persistConn).readLoop(0xc420a46360) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by net/http.(*Transport).dialConn Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 164 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*persistConn).writeLoop(0xc420a46360) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by net/http.(*Transport).dialConn Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 217 [select, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4208d7140, 0xc4212eb2c8, 0x20, 0x20, 0x5, 0x1bd9a98) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42040d7d0, 0x1d718c0, 0xc4208d7140, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42048f680, 0xc4212eb500, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Node).List(0xc42034e5a0, 0xc4208d64e0, 0xc42040a990, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:1198 +0x27c Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: reflect.Value.call(0xc42034e8a0, 0xc420226d60, 0x13, 0x1bda9a0, 0x4, 0xc4210938f8, 0x3, 0x3, 0x192ba60, 0x19, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: reflect.Value.Call(0xc42034e8a0, 0xc420226d60, 0x13, 0xc4204ce8f8, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*service).call(0xc420220a80, 0xc42020f450, 0xc42068e0c0, 0x0, 0xc420135100, 0xc4207821c0, 0x1975760, 0xc4208d64e0, 0x16, 0x169e120, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc42020f450, 0x1d71b80, 0xc420da6580, 0xc4201e8800, 0xa) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc42048f680, 0x1be205d, 0x9, 0x1975760, 0xc4208d6480, 0x169e120, 0xc42040a930, 0xc4208d6401, 0xc42040a930) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc420378070, 0x1be205d, 0x9, 0x1975760, 0xc4208d6480, 0x169e120, 0xc42040a930, 0xc42013f260, 0x1e) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest(0xc420e2c340, 0x1d6df80, 0xc4208d6420, 0xc420e96500, 0xc42013f290, 0xc4204ceb58, 0x38, 0xc4204ceb50) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/node_endpoint.go:24 +0x1b3 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest-fm(0x1d6df80, 0xc4208d6420, 0xc420e96500, 0xc420e96500, 0xc4206d92c0, 0x1e, 0xbed7c7d03acc3f8e) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:145 +0x48 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc4208d6420, 0xc420e96500) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.HandlerFunc.ServeHTTP(0xc420749580, 0x1d6df80, 0xc4208d6420, 0xc420e96500) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*ServeMux).ServeHTTP(0xc420fce120, 0x1d6df80, 0xc4208d6420, 0xc420e96500) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc420b070a0, 0xc420e96500) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.HandlerFunc.ServeHTTP(0xc420fce2a0, 0x1d6fec0, 0xc420b070a0, 0xc420e96500) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.serverHandler.ServeHTTP(0xc4201ba5b0, 0x1d6fec0, 0xc420b070a0, 0xc420e96500) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*conn).serve(0xc420386a00, 0x1d71840, 0xc420da62c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by net/http.(*Server).Serve Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 218 [IO wait, 1 minutes]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676610, 0x72, 0xc421053e58) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420823818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420823818, 0xc4205dbe00, 0x1, 0x1) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc420823800, 0xc4205dbed1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc420823800, 0xc4205dbed1, 0x1, 0x1, 0xc, 0xc, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420216540, 0xc4205dbed1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/http.(*connReader).backgroundRead(0xc4205dbec0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 231 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421572370) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 246 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421572370, 0xc420066d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421572370, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420216998, 0xc4207a0050) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420216998, 0xc4207a0050) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 209 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676540, 0x72, 0xc421255ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc42232bf18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc42232bf18, 0xc42251c000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc42232bf00, 0xc42251c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc42232bf00, 0xc42251c000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420216bc0, 0xc42251c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc42201e4e0, 0xc42259d3d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc42201e4e0, 0xc42259d3d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc42201e4e0, 0xc42259d3d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4215736b0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4215736b0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 438 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420588c80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 258 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4215736b0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 259 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4215736b0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 234 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421ebfad0, 0xc4204cfd10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421ebfad0, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420276dd0, 0xc4210314f0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420276dd0, 0xc4210314f0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 235 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e6763a0, 0x72, 0xc421253ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc42232a018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc42232a018, 0xc420f29000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc42232a000, 0xc420f29000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc42232a000, 0xc420f29000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420276dd0, 0xc420f29000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc4225ebb00, 0xc4223f5e30, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc4225ebb00, 0xc4223f5e30, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc4225ebb00, 0xc4223f5e30, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421ebfad0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421ebfad0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 236 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421ebfad0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 237 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421ebfad0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 238 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204eea80, 0xc420be4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc420f95ec0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc420f95ec0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4217d3050, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421448a00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420f95f20, 0x16e4b60, 0xc421448a60) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420f95f20, 0x16e4b60, 0xc421448a60, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421c5d3c0, 0x16e4b60, 0xc421448a60, 0xc421448a60, 0xc421031608) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421c5d3c0, 0xc421448a60, 0x2d, 0xc4217c7fc0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4210315e0, 0x1d72340, 0xc421c5d3c0, 0x30, 0x1, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4210315e0, 0x1d72340, 0xc421c5d3c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4210315e0, 0x1d72340, 0xc421c5d3c0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc4204eea80, 0xc4210315e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 239 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421e52630, 0xc421252d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421e52630, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42000f1d0, 0xc4212c0410) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42000f1d0, 0xc4212c0410) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 247 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676200, 0x72, 0xc4204cece8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc42170b298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc42170b298, 0xc42152b000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc42170b280, 0xc42152b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc42170b280, 0xc42152b000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420227488, 0xc42152b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc42256e840, 0xc42242da30, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc42256e840, 0xc42242da30, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc42256e840, 0xc42242da30, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420457b80, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420457b80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 248 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420457b80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 264 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420f9fe40, 0xc4211ead10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420f9fe40, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42000eda8, 0xc4207a06e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42000eda8, 0xc4207a06e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 275 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420f9f970) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 276 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204eee40, 0xc421979000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc4227c4c60) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc4227c4c60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421eee9c0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4219baa20) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4227c4cc0, 0x16e4b60, 0xc4219baaa0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4227c4cc0, 0x16e4b60, 0xc4219baaa0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4224a8d40, 0x16e4b60, 0xc4219baaa0, 0xc4219baaa0, 0xc420e0a898) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4224a8d40, 0xc4219baaa0, 0x2d, 0xc4227e6660) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc420e0a870, 0x1d72340, 0xc4224a8d40, 0x30, 0x1, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc420e0a870, 0x1d72340, 0xc4224a8d40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc420e0a870, 0x1d72340, 0xc4224a8d40, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc4204eee40, 0xc420e0a870) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 277 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420457b80, 0xc4211e8d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420457b80, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420227488, 0xc4215f2ff0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420227488, 0xc4215f2ff0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 250 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc42048f680, 0x1bef5ff, 0x11, 0x1d77180, 0xc421f81320, 0x1975b20, 0xc421f81320, 0x169e1e0, 0xc42273eb80, 0xc422698810, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Node).UpdateStatus(0xc421a38960, 0xc421f81320, 0xc42273eb80, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:321 +0xa9 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: reflect.Value.call(0xc42034ea80, 0xc42000ea30, 0x13, 0x1bda9a0, 0x4, 0xc421091d48, 0x3, 0x3, 0x1a99fe0, 0x19, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: reflect.Value.Call(0xc42034ea80, 0xc42000ea30, 0x13, 0xc421091d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*service).call(0xc4221a1e80, 0xc420b843c0, 0xc421826108, 0x0, 0xc420c00200, 0xc420bfaf80, 0x1975b20, 0xc421f81320, 0x16, 0x169e1e0, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc420b843c0, 0x1d72340, 0xc4221a1ec0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc422736600, 0xc420b843c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 252 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420244c00, 0xc42221e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc42138b8c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc42138b8c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420e52930, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420681160) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42138b920, 0x16e4b60, 0xc4206811c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42138b920, 0x16e4b60, 0xc4206811c0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420a86980, 0x16e4b60, 0xc4206811c0, 0xc4206811c0, 0xc4207a07a8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420a86980, 0xc4206811c0, 0x2d, 0xc4211081e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4207a0780, 0x1d72340, 0xc420a86980, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4207a0780, 0x1d72340, 0xc420a86980, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4207a0780, 0x1d72340, 0xc420a86980, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc420244c00, 0xc4207a0780) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 265 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676130, 0x72, 0xc42281dce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc4226c4f98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc4226c4f98, 0xc4220b0000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc4226c4f80, 0xc4220b0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc4226c4f80, 0xc4220b0000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc42000eda8, 0xc4220b0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc420f34720, 0xc421a1ec10, 0xc, 0xc, 0x3, 0x40df9e, 0xc42239cc00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc420f34720, 0xc421a1ec10, 0xc, 0xc, 0xc, 0xc, 0xc4216f75c0, 0xc420f9fe84) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc420f34720, 0xc421a1ec10, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420f9fe40, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420f9fe40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 266 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420f9fe40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 267 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420f9fe40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 240 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676060, 0x72, 0xc42281bce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc4211cf418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc4211cf418, 0xc422014000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc4211cf400, 0xc422014000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc4211cf400, 0xc422014000, 0x1000, 0x1000, 0xc42281bdd8, 0x0, 0xc42281be08) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc42000f1d0, 0xc422014000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc421ae4c60, 0xc42235e174, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc421ae4c60, 0xc42235e174, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc421ae4c60, 0xc42235e174, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421e52630, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421e52630) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 241 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421e52630) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 290 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421e52630) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 425 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4202a6840, 0xc420beb2c0, 0xc423099440) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a6840, 0xc421958540) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 301 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e623d50, 0x72, 0xc422d6cce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc421453c98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc421453c98, 0xc422e84000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc421453c80, 0xc422e84000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc421453c80, 0xc422e84000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420217a48, 0xc422e84000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc42201fbc0, 0xc4228170d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc42201fbc0, 0xc4228170d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc42201fbc0, 0xc4228170d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421ebf550, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421ebf550) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 302 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421ebf550) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 297 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42114c000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 298 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc422736cc0, 0xc42234e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc420905da0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc420905da0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420ca55f0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206803e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420905e00, 0x16e4b60, 0xc420680460) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420905e00, 0x16e4b60, 0xc420680460, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421e449c0, 0x16e4b60, 0xc420680460, 0xc420680460, 0xc4205d4118) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421e449c0, 0xc420680460, 0x2d, 0xc42117bb40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4205d40f0, 0x1d72340, 0xc421e449c0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4205d40f0, 0x1d72340, 0xc421e449c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4205d40f0, 0x1d72340, 0xc421e449c0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc422736cc0, 0xc4205d40f0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 288 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42024ca50) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 289 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204ef140, 0xc42254d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc4215ddb60) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc4215ddb60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42128e120, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420482940) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4215ddbc0, 0x16e4b60, 0xc4204829a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4215ddbc0, 0x16e4b60, 0xc4204829a0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4212840c0, 0x16e4b60, 0xc4204829a0, 0xc4204829a0, 0xc4200a9fb8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4212840c0, 0xc4204829a0, 0x2d, 0xc420a50f40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4200a9f90, 0x1d72340, 0xc4212840c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4200a9f90, 0x1d72340, 0xc4212840c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4200a9f90, 0x1d72340, 0xc4212840c0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc4204ef140, 0xc4200a9f90) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 286 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e624230, 0x72, 0xc422dffce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc42273f818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc42273f818, 0xc42254c000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc42273f800, 0xc42254c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc42273f800, 0xc42254c000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420216fe0, 0xc42254c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc4209013e0, 0xc420aecb20, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc4209013e0, 0xc420aecb20, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc4209013e0, 0xc420aecb20, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42024ca50, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42024ca50) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 287 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42024ca50) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 329 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e624300, 0x72, 0xc422701ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc421affe18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc421affe18, 0xc421f8b000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc421affe00, 0xc421f8b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc421affe00, 0xc421f8b000, 0x1000, 0x1000, 0xc42108bdd8, 0x1, 0xc42108be08) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc4202167a8, 0xc421f8b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc420d62a80, 0xc421da6af4, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc420d62a80, 0xc421da6af4, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc420d62a80, 0xc421da6af4, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420f9fa20, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420f9fa20) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 285 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42024ca50, 0xc422705d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42024ca50, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420216fe0, 0xc4200a9ef0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420216fe0, 0xc4200a9ef0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 328 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420f9fa20, 0xc422702d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420f9fa20, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc4202167a8, 0xc420b85f90) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc4202167a8, 0xc420b85f90) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 295 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e623ef0, 0x72, 0xc422d6fce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc4219ae998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc4219ae998, 0xc422387000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc4219ae980, 0xc422387000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc4219ae980, 0xc422387000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420217388, 0xc422387000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc42105a780, 0xc42021aa10, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc42105a780, 0xc42021aa10, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc42105a780, 0xc42021aa10, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42114c000, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42114c000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 296 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42114c000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 323 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421323080, 0xc422818d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421323080, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42198a148, 0xc4203d1e00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42198a148, 0xc4203d1e00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 324 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e6244a0, 0x72, 0xc422706ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420bedc98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420bedc98, 0xc420f44000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc420bedc80, 0xc420f44000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc420bedc80, 0xc420f44000, 0x1000, 0x1000, 0xac, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc42198a148, 0xc420f44000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc42105b920, 0xc4218a56c4, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc42105b920, 0xc4218a56c4, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc42105b920, 0xc4218a56c4, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421323080, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421323080) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 325 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421323080) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 326 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421323080) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 327 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc422736e40, 0xc420f45000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc420a4f620) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc420a4f620, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420e349c0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420681f00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420a4f680, 0x16e4b60, 0xc420681f60) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420a4f680, 0x16e4b60, 0xc420681f60, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421f7e240, 0x16e4b60, 0xc420681f60, 0xc420681f60, 0xc4203d1f68) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421f7e240, 0xc420681f60, 0x2d, 0xc421c93260) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4203d1f40, 0x1d72340, 0xc421f7e240, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4203d1f40, 0x1d72340, 0xc421f7e240, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4203d1f40, 0x1d72340, 0xc421f7e240, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc422736e40, 0xc4203d1f40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 311 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421322c60, 0xc42281ad10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421322c60, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420227588, 0xc4203c4cd0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420227588, 0xc4203c4cd0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 312 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e6243d0, 0x72, 0xc422703ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc4217c3718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc4217c3718, 0xc421a22000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc4217c3700, 0xc421a22000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc4217c3700, 0xc421a22000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420227588, 0xc421a22000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc422385680, 0xc421fe9294, 0xc, 0xc, 0x3, 0x40df9e, 0xc42159a8a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc422385680, 0xc421fe9294, 0xc, 0xc, 0xc, 0xc, 0xc423099800, 0xc421322ca4) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc422385680, 0xc421fe9294, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421322c60, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421322c60) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 313 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421322c60) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 314 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421322c60) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 315 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc42048f680, 0x1bef5ff, 0x11, 0x1d77180, 0xc421f81560, 0x1975b20, 0xc421f81560, 0x169e1e0, 0xc42273ec00, 0xc422698810, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Node).UpdateStatus(0xc421b478c0, 0xc421f81560, 0xc42273ec00, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:321 +0xa9 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: reflect.Value.call(0xc42034ea80, 0xc420227c98, 0x13, 0x1bda9a0, 0x4, 0xc421097d48, 0x3, 0x3, 0x1a99fe0, 0x19, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: reflect.Value.Call(0xc42034ea80, 0xc420227c98, 0x13, 0xc421097d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*service).call(0xc4221a0a00, 0xc4203c4f00, 0xc422323c10, 0x0, 0xc42070a580, 0xc4203f42c0, 0x1975b20, 0xc421f81560, 0x16, 0x169e1e0, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4203c4f00, 0x1d72340, 0xc4221a0a40, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc4204eec00, 0xc4203c4f00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 330 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420f9fa20) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 331 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420f9fa20) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 332 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204eef00, 0xc4209a4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc420b19a40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc420b19a40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421754810, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420bdffc0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420b19c80, 0x16e4b60, 0xc420722020) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420b19c80, 0x16e4b60, 0xc420722020, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421cca140, 0x16e4b60, 0xc420722020, 0xc420722020, 0xc42039a1b8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421cca140, 0xc420722020, 0x2d, 0xc421db02c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc42039a190, 0x1d72340, 0xc421cca140, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc42039a190, 0x1d72340, 0xc421cca140, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc42039a190, 0x1d72340, 0xc421cca140, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc4204eef00, 0xc42039a190) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 334 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4215733f0, 0xc4214bdd10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4215733f0, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42198ac68, 0xc420d547d0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42198ac68, 0xc420d547d0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 335 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e624160, 0x72, 0xc420065ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc421849398, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc421849398, 0xc421e1a000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc421849380, 0xc421e1a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc421849380, 0xc421e1a000, 0x1000, 0x1000, 0xc42108f5d8, 0x1, 0xc42108f608) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc42198ac68, 0xc421e1a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc42267ae40, 0xc4217afd60, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc42267ae40, 0xc4217afd60, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc42267ae40, 0xc4217afd60, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4215733f0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4215733f0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 336 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4215733f0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 337 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4215733f0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 338 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204ef380, 0xc421e1b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc420f953e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc420f953e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421ab8ae0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207835e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420f95440, 0x16e4b60, 0xc420783640) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420f95440, 0x16e4b60, 0xc420783640, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421e22e00, 0x16e4b60, 0xc420783640, 0xc420783640, 0xc420d54898) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421e22e00, 0xc420783640, 0x2d, 0xc42232e060) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc420d54870, 0x1d72340, 0xc421e22e00, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc420d54870, 0x1d72340, 0xc421e22e00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc420d54870, 0x1d72340, 0xc421e22e00, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc4204ef380, 0xc420d54870) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 354 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421ebf760, 0xc42281fd10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421ebf760, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42198b190, 0xc420e0aeb0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42198b190, 0xc420e0aeb0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 355 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e624090, 0x72, 0xc422dfdce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc4219e3818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc4219e3818, 0xc4212d9000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc4219e3800, 0xc4212d9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc4219e3800, 0xc4212d9000, 0x1000, 0x1000, 0xc42108d5d8, 0x1, 0xc42108d608) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc42198b190, 0xc4212d9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc4217527e0, 0xc4223376d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc4217527e0, 0xc4223376d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc4217527e0, 0xc4223376d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421ebf760, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421ebf760) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 356 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421ebf760) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 357 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421ebf760) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 358 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420244fc0, 0xc421006000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc421b0cd20) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc421b0cd20, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4223a8d20, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420639120) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421b0cd80, 0x16e4b60, 0xc420639200) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421b0cd80, 0x16e4b60, 0xc420639200, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4224463c0, 0x16e4b60, 0xc420639200, 0xc420639200, 0xc420e0b018) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4224463c0, 0xc420639200, 0x2d, 0xc42269c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc420e0aff0, 0x1d72340, 0xc4224463c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc420e0aff0, 0x1d72340, 0xc4224463c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc420e0aff0, 0x1d72340, 0xc4224463c0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc420244fc0, 0xc420e0aff0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 451 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4202a6840, 0xc420beb2c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:123 +0x1fd Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a6840, 0xc420bfa8c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 340 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4204574a0, 0xc422700d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4204574a0, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420276880, 0xc4212c03c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420276880, 0xc4212c03c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 341 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e623fc0, 0x72, 0xc4201b7ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420bec298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420bec298, 0xc4225d6000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc420bec280, 0xc4225d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc420bec280, 0xc4225d6000, 0x1000, 0x1000, 0xc4201b7dd8, 0x0, 0xc4201b7e08) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420276880, 0xc4225d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc4212b8c00, 0xc421b7b094, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc4212b8c00, 0xc421b7b094, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc4212b8c00, 0xc421b7b094, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4204574a0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4204574a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 342 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4204574a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 343 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4204574a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 399 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4213228f0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 317 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42114c000, 0xc422dfed10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42114c000, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420217388, 0xc42039ba40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420217388, 0xc42039ba40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 319 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4218b06e0, 0xc42281cd10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4218b06e0, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42000f1b8, 0xc4225d4550) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc42000f1b8, 0xc4225d4550) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 320 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e623e20, 0x72, 0xc422707ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420e73018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420e73018, 0xc421a4f000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc420e73000, 0xc421a4f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc420e73000, 0xc421a4f000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc42000f1b8, 0xc421a4f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc42267a600, 0xc421cc13c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc42267a600, 0xc421cc13c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc42267a600, 0xc421cc13c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4218b06e0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4218b06e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 321 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4218b06e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 370 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4218b06e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 371 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4211a2900, 0xc42241d700, 0x20, 0x20, 0x169d4e0, 0xc4206cafa0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224a0d20, 0x1d718c0, 0xc4211a2900, 0x0, 0xc4211a2900) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42048f680, 0xc42241d950, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420f94720, 0xc421848080, 0xc4224a0ab0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: reflect.Value.call(0xc42034e7e0, 0xc42000f818, 0x13, 0x1bda9a0, 0x4, 0xc42241dd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: reflect.Value.Call(0xc42034e7e0, 0xc42000f818, 0x13, 0xc42241dd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*service).call(0xc421e31400, 0xc4225d45f0, 0xc420c8c4b8, 0x0, 0xc422d05a80, 0xc4211cd400, 0x19758e0, 0xc421848080, 0x16, 0x169e020, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4225d45f0, 0x1d72340, 0xc421e316c0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc420bea540, 0xc4225d45f0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 372 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421ebf550, 0xc422d6ed10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421ebf550, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420217a48, 0xc42096bb80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420217a48, 0xc42096bb80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 303 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421ebf550) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 304 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204ef2c0, 0xc422e85000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc4223905a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc4223905a0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4226985a0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422372ea0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422390600, 0x16e4b60, 0xc422372f00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422390600, 0x16e4b60, 0xc422372f00, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42294ed00, 0x16e4b60, 0xc422372f00, 0xc422372f00, 0xc4225d5ec8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42294ed00, 0xc422372f00, 0x2d, 0xc421c939c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4225d5ea0, 0x1d72340, 0xc42294ed00, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4225d5ea0, 0x1d72340, 0xc42294ed00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4225d5ea0, 0x1d72340, 0xc42294ed00, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc4204ef2c0, 0xc4225d5ea0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 454 [runnable]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: syscall.Syscall(0x0, 0x25, 0xc421ab4000, 0x1000, 0xffffffffffffffff, 0x0, 0xb) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: syscall.read(0x25, 0xc421ab4000, 0x1000, 0x1000, 0xc420da6700, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/syscall/zsyscall_linux_amd64.go:749 +0x5f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: syscall.Read(0x25, 0xc421ab4000, 0x1000, 0x1000, 0x86, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/syscall/syscall_unix.go:162 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc42273f080, 0xc421ab4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:153 +0x118 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc42273f080, 0xc421ab4000, 0x1000, 0x1000, 0x86, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420277930, 0xc421ab4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc421f81d40, 0xc422042380, 0x292, 0x30c, 0x1b70ac0, 0x7fc80e5b2020, 0xc422bf2be7) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.(*LimitedReader).Read(0xc421959360, 0xc422042380, 0x30c, 0x30c, 0xc421e32480, 0xc420040000, 0xc422bf2c68) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:446 +0x63 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bytes.(*Buffer).ReadFrom(0xc4222e4c40, 0x1d5bf40, 0xc421959360, 0x7fc80e5b2020, 0xc4222e4c40, 0xc422bf2c01) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bytes/buffer.go:205 +0xa0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.copyBuffer(0x1d59b20, 0xc4222e4c40, 0x1d5bf40, 0xc421959360, 0x0, 0x0, 0x0, 0x1944e20, 0x1, 0xc421959360) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:386 +0x31a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.Copy(0x1d59b20, 0xc4222e4c40, 0x1d5bf40, 0xc421959360, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:362 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).readData(0xc422737440, 0xc421c735e0, 0xc, 0xc, 0x0, 0x1d59aa0, 0xc421f81d40, 0xc422bf2eb0, 0x466056) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:423 +0x148 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).handleStreamMessage(0xc42114c9a0, 0xc421c735e0, 0xc, 0xc, 0xc, 0xc) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:518 +0x29b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42114c9a0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:469 +0x176 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42114c9a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 390 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421e53c30, 0xc422819d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421e53c30, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420277b00, 0xc420187400) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420277b00, 0xc420187400) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 453 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42114c9a0, 0xc422d6dd10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42114c9a0, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420277930, 0xc4215ed630) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420277930, 0xc4215ed630) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 393 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421e53c30) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 394 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc422737500, 0xc420bff000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc42186daa0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc42186daa0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422220330, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4211117e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42186db00, 0x16e4b60, 0xc421111860) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42186db00, 0x16e4b60, 0xc421111860, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422888d40, 0x16e4b60, 0xc421111860, 0xc421111860, 0xc4201874c8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422888d40, 0xc421111860, 0x2d, 0xc421c92360) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4201874a0, 0x1d72340, 0xc422888d40, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4201874a0, 0x1d72340, 0xc422888d40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4201874a0, 0x1d72340, 0xc422888d40, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc422737500, 0xc4201874a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 391 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e623c80, 0x72, 0xc42285cce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc420c14a98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc420c14a98, 0xc420bfe000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc420c14a80, 0xc420bfe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc420c14a80, 0xc420bfe000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420277b00, 0xc420bfe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc421a39320, 0xc422124ba0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc421a39320, 0xc422124ba0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc421a39320, 0xc422124ba0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421e53c30, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421e53c30) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 392 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421e53c30) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 452 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4202a6840, 0xc420beb380, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4202a6840, 0xc420beb380) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a6840, 0xc420bfa8e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 443 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc4224a1920) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 365 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421e52790) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 366 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421e52790) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 439 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc42020e410, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc42048f680) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 400 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420beaf00, 0xc4222ca000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc421b477a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc421b477a0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4225032f0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421987200) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421b47800, 0x16e4b60, 0xc421987260) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421b47800, 0x16e4b60, 0xc421987260, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422b39d40, 0x16e4b60, 0xc421987260, 0xc421987260, 0xc4215f31a8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422b39d40, 0xc421987260, 0x2d, 0xc422325ea0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4215f3180, 0x1d72340, 0xc422b39d40, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4215f3180, 0x1d72340, 0xc422b39d40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4215f3180, 0x1d72340, 0xc422b39d40, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc420beaf00, 0xc4215f3180) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 424 [runnable]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldBlock(0xc421c0f600, 0x1a1c000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6607 +0x2a4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc42048f680, 0xc420229501, 0xc420229560) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:300 +0x152 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc42048f680, 0xc4230993e0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc42048f680, 0xc4230993e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420b32110, 0xc42048f680, 0xc4230993e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 397 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e676880, 0x72, 0xc422dfcce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc4207c9418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc4207c9418, 0xc421fc3000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc4207c9400, 0xc421fc3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc4207c9400, 0xc421fc3000, 0x1000, 0x1000, 0xc4200515d8, 0x1, 0xc420051608) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420216bb0, 0xc421fc3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc422170c60, 0xc421c73430, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc422170c60, 0xc421c73430, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc422170c60, 0xc421c73430, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4213228f0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4213228f0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 398 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4213228f0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 396 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4213228f0, 0xc4201b0d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4213228f0, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420216bb0, 0xc4215f3090) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420216bb0, 0xc4215f3090) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 415 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421e529a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 413 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421e529a0, 0xc422858d10, 0x30, 0xf9866a) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421e529a0, 0x1c61988, 0xc42048f680, 0x1d71840, 0xc420e2c100) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420277558, 0xc4215ece10) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d320, 0xc420277558, 0xc4215ece10) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 414 [IO wait]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.runtime_pollWait(0x7fc80e623bb0, 0x72, 0xc4211e9ce8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).wait(0xc4209fec98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*pollDesc).waitRead(0xc4209fec98, 0xc420cc8000, 0x1000, 0x1000) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: internal/poll.(*FD).Read(0xc4209fec80, 0xc420cc8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*netFD).Read(0xc4209fec80, 0xc420cc8000, 0x1000, 0x1000, 0xc420050dd8, 0x1, 0xc420050e08) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net.(*conn).Read(0xc420277558, 0xc420cc8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).Read(0xc42186c900, 0xc421c735b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadAtLeast(0x1d59aa0, 0xc42186c900, 0xc421c735b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: io.ReadFull(0x1d59aa0, 0xc42186c900, 0xc421c735b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421e529a0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421e529a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 417 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420beafc0, 0xc420cc9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc421f81080) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc421f81080, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4224e9c50, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4219f5f80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421f810e0, 0x16e4b60, 0xc4219f5fe0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421f810e0, 0x16e4b60, 0xc4219f5fe0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422acb2c0, 0x16e4b60, 0xc4219f5fe0, 0xc4219f5fe0, 0xc4215ecf78) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422acb2c0, 0xc4219f5fe0, 0x2d, 0xc422863f00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4215ecf50, 0x1d72340, 0xc422acb2c0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4215ecf50, 0x1d72340, 0xc422acb2c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4215ecf50, 0x1d72340, 0xc422acb2c0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc420beafc0, 0xc4215ecf50) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 427 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4202a6840, 0xc420beb380, 0xc4230995c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a6840, 0xc4219585c0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 455 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42114c9a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 456 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42114c9a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 437 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420588c80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 428 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc422fe3500) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 429 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4202a6840, 0xc420beb380, 0x1d6eb80, 0xc422fe3500, 0xc423099680, 0xc4230996e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a6840, 0xc422fe3540) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 432 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc422737440, 0xc421b30000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).fill(0xc421ed9d40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: bufio.(*Reader).ReadByte(0xc421ed9d40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4224d2ed0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421958b80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421ed9da0, 0x16e4b60, 0xc421958be0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421ed9da0, 0x16e4b60, 0xc421958be0, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422fe3c80, 0x16e4b60, 0xc421958be0, 0xc421958be0, 0xc4215ed6f8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422fe3c80, 0xc421958be0, 0x2d, 0xc421c93540) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequestHeader(0xc4215ed6d0, 0x1d72340, 0xc422fe3c80, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42039a0f0, 0x8) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).readRequest(0xc4215ed6d0, 0x1d72340, 0xc422fe3c80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: net/rpc.(*Server).ServeRequest(0xc4215ed6d0, 0x1d72340, 0xc422fe3c80, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048f680, 0x1d71840, 0xc420e2c100, 0x1d7d0e0, 0xc422737440, 0xc4215ed6d0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 466 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4224d3890, 0xc423099c20) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 444 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc421fcd740, 0xc422beece8, 0x20, 0x20, 0x18, 0xc4219590e0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224d3b30, 0x1d71840, 0xc421fcd740, 0xc4219590e0, 0x184987) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420229560, 0xc422b29ed0, 0x184987, 0x1d71840, 0xc421fcd740, 0xc42070b860, 0xc4224a18f0, 0xc420c6e750, 0x24, 0x2bba400) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc42070b800, 0x1d71840, 0xc421fcd740, 0x184987, 0xc421f96440, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc42070b800, 0x1d71840, 0xc421fcd740) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 445 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc421fcda80, 0xc422ab9770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422429740, 0x1d71840, 0xc421fcda80, 0x0, 0x1849b5) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420229560, 0xc422ab9960, 0x1849b5, 0x1d71840, 0xc421fcda80, 0x552bde, 0xc422ab9950, 0xc422ab9970, 0xf33ba5, 0xc420569088) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc420569030, 0x1d71840, 0xc421fcda80, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc420569030) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 446 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc421fcd780, 0xc422ab5a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422429560, 0x1d71840, 0xc421fcd780, 0xc422429680, 0x1833a4) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420229560, 0xc422ab5c68, 0x1833a4, 0x1d71840, 0xc421fcd780, 0x552bde, 0xc42020e140, 0x2, 0xc42033f880, 0x40) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc421fcdb80, 0x1833a4, 0x3b, 0xc422bf3de8, 0x1, 0x1) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc421fcdb80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 467 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4224d3890, 0xc423099c80) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 447 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc421fcddc0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 448 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420c665a0, 0x1d71840, 0xc421fcd780) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 468 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc420fac0a0) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: goroutine 470 [select]: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc421f96680, 0xc422befd28, 0x20, 0x20, 0xc4201fec00, 0x1bd9a98) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224d3ef0, 0x1d71840, 0xc421f96680, 0xc421959340, 0x1849b5) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420229560, 0xc422b93f20, 0x1849b5, 0x1d71840, 0xc421f96680, 0xf28cc9, 0xc42070b800, 0x1d71840, 0xc421fcd740, 0x1) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc420fac0a0, 0x1849b5, 0x0, 0x0, 0x0, 0x0, 0xf9897b, 0x3) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc420fac0a0, 0x1849b5, 0xc42120c600) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25353]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:26:42 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:26:42 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:26:42 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: ==> Starting Nomad agent... Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: ==> Nomad agent configuration: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: Client: false Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: Log Level: INFO Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: Region: amrv01 (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: Server: true Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: Version: 0.8.4 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.470576 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: Attempting re-join to previously known node: consulnomad-staging-01.asv01: 172.27.0.136:4648 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.470770 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [WARN] memberlist: Refuting an alive message Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825027 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825051 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825065 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825078 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825091 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825335 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825352 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825365 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825376 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825387 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825550 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825570 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825584 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825596 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825712 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825729 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825741 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825753 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42 [INFO] serf: Re-joined to previously known node: consulnomad-staging-01.asv01: 172.27.0.136:4648 Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825876 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:26:42 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:42.825890 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:26:43 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:43 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:43 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:43 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:43 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:43 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:43 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:43 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:43 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:43 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:43 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:43 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:43 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:43 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:43 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:44 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:44 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:44 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:44 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:45 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:45.037402 [ERR] http: Request /v1/agent/health?type=server, error: {"server":{"ok":false,"message":"rpc error: EOF"}} Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 15 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [INFO] raft: Election won. Tally: 2 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46.089273 [INFO] nomad: cluster leadership acquired Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: 2018/08/23 15:26:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: fatal error: concurrent map writes Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 290 [running]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc422a61d08 sp=0xc422a61ce8 pc=0x42d161 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: runtime.mapdelete(0x1844be0, 0xc4201b8fc0, 0xc422a61e20) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc422a61d68 sp=0xc422a61d08 pc=0x40a761 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc420010bc0, 0x1d6dd40, 0xc42000f1e0, 0x0, 0xed3109f06) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc422a61e50 sp=0xc422a61d68 pc=0xf2164c Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc420334f00, 0x1d71840, 0xc422bf56c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc422a61fc8 sp=0xc422a61e50 pc=0xf56bb7 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: runtime.goexit() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc422a61fd0 sp=0xc422a61fc8 pc=0x45c711 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 1 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420234a10, 0xc4203e5680) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420234a10, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4200ef680, 0xc4200ef680, 0xc4204a00c0, 0x37) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204e1f68) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: main.main() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 5 [syscall]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: os/signal.signal_recv(0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: os/signal.loop() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by os/signal.init.0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 127 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753880, 0x72, 0xc42117ace8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc42099e198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc42099e198, 0xc420bf6000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc42099e180, 0xc420bf6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc42099e180, 0xc420bf6000, 0x1000, 0x1000, 0xc42117add8, 0x1, 0xc42117ae08) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc42020c038, 0xc420bf6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc4213f91a0, 0xc4204dcae0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc4213f91a0, 0xc4204dcae0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc4213f91a0, 0xc4204dcae0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200be2c0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200be2c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 177 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421ec8000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 128 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200be2c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 50 [select, locked to thread]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: runtime.selectgo(0xc420050750, 0xc420090720) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: runtime.ensureSigM.func1() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: runtime.goexit() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 51 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42042b380) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 52 [sleep]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: time.Sleep(0x3b9aca00) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420243a20) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 53 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420298540) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 54 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4201f4d80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 108 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420ef8090) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 56 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc420380680) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4201b9410, 0xc4201fbfb0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 27 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc42042b440, 0x1d71840, 0xc420212e80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 125 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202433f0, 0xc4210a6d10, 0x30, 0xf9866a) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202433f0, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42020c030, 0xc420da23c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42020c030, 0xc420da23c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 106 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200b42c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc42103bda0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 110 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753d60, 0x72, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc420134298, 0x72, 0xc420212f00, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc420134298, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Accept(0xc420134280, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).accept(0xc420134280, 0x0, 0xc4204b27a8, 0x721378) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*TCPListener).accept(0xc42000e600, 0x0, 0xc4202e82a1, 0x5) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*TCPListener).AcceptTCP(0xc42000e600, 0x1, 0x0, 0x435248) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420ebe0f0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 292 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420296b40) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 109 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420d40410) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 59 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753f00, 0x72, 0xc4205cb9a8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc4204e6818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc4204e6818, 0xc420377000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc4204e6800, 0xc420377000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc4204e6800, 0xc420377000, 0x1000, 0x1000, 0x458600, 0xc4201e9380, 0x4) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc42020cf08, 0xc420377000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/http.(*persistConn).Read(0xc4204025a0, 0xc420377000, 0x1000, 0x1000, 0xc4205cbb98, 0x405675, 0xc420504ba0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).fill(0xc420446d80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Peek(0xc420446d80, 0x1, 0x0, 0x0, 0x1, 0xc420dba9c0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/http.(*persistConn).readLoop(0xc4204025a0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by net/http.(*Transport).dialConn Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 126 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200be2c0, 0xc42117ed10, 0x30, 0xf9866a) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200be2c0, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42020c038, 0xc420da24b0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42020c038, 0xc420da24b0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 45 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420301400, 0x1c636a0, 0xc4204fc0b0, 0x1d71840, 0xc420591ec0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4204fc0b0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 60 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/http.(*persistConn).writeLoop(0xc4204025a0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by net/http.(*Transport).dialConn Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 223 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422d02420) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 64 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c737536e0, 0x72, 0xc42117dce8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc420be2e98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc420be2e98, 0xc420a6e000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc420be2e80, 0xc420a6e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc420be2e80, 0xc420a6e000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc42056c7f0, 0xc420a6e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc420c57da0, 0xc4206fe000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc420c57da0, 0xc4206fe000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc420c57da0, 0xc4206fe000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420243550, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420243550) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 222 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422d02420) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 107 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200b42c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc42103bdb0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 105 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200b42c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200b42c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200b42c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc42103bd90) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:46 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 111 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753c90, 0x72, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc420cc4118, 0x72, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc420cc4118, 0xc422ff8000, 0x10000, 0x10000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).ReadFrom(0xc420cc4100, 0xc422ff8000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).readFrom(0xc420cc4100, 0xc422ff8000, 0x10000, 0x10000, 0x344, 0x344, 0x1d62b40, 0x412539, 0xc421101e40) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*UDPConn).readFrom(0xc42000e618, 0xc422ff8000, 0x10000, 0x10000, 0x101, 0xc421101e18, 0xc421101e90, 0x443cf7) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*UDPConn).ReadFrom(0xc42000e618, 0xc422ff8000, 0x10000, 0x10000, 0x10000, 0x10000, 0x1637d, 0xe412a4cf, 0x2b96de0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420ebe0f0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 112 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420ebe0f0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 113 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420ebe0f0, 0x12a05f200, 0xc42001ad80, 0xc420e30180, 0xc42079e340) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 130 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420ebe0f0, 0xc420e30180) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 131 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420ebe0f0, 0x1dcd6500, 0xc42001ade0, 0xc420e30180, 0xc42079e350) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 132 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4201e8600) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 133 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4201e8600) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 134 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4201e8600, 0x1bdd09a, 0x6, 0xc420ef81e0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 135 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4201e8600, 0x1bdba6c, 0x5, 0xc420ef8210) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 136 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4201e8600, 0x1bdbc6a, 0x5, 0xc420ef8240) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 270 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc421074000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 138 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc42001ae40, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc42001ae40, 0x1dcd6500, 0x0, 0x0, 0x0, 0xc420494240, 0x71a2da) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc42001ae40) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 139 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc42001aea0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc42001aea0, 0x1dcd6500, 0x63c900, 0x1, 0x0, 0x435248, 0x1c66060) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc42001aea0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 140 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc42001af00, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc42001af00, 0x1dcd6500, 0x0, 0xc420e30180, 0xa10b06, 0x1, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc42001af00) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 141 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc42001af60, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc42001af60, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc42001af60) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 142 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc421440240, 0xc4200e1b80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 143 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4200e1b80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 144 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4200e1b80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 145 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753e30, 0x72, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc4204e7418, 0x72, 0xc420212c00, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc4204e7418, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Accept(0xc4204e7400, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).accept(0xc4204e7400, 0x1, 0x1, 0xc420067f20) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*TCPListener).accept(0xc420202530, 0x406b4a, 0xc420dac060, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*TCPListener).Accept(0xc420202530, 0xc420dac060, 0x0, 0x3f800000, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 146 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc420334f00, 0x3b9aca00, 0xc420090ba0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 147 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4201ff5e0, 0x3b9aca00, 0xc420090ba0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 148 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420296b40, 0x3b9aca00, 0xc420090ba0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 149 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc420380680, 0x3b9aca00, 0xc420090ba0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 150 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4200e1b80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 151 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753bc0, 0x72, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc4204e6098, 0x72, 0xc420c8e000, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc4204e6098, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Accept(0xc4204e6080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).accept(0xc4204e6080, 0x402037, 0xc42080ff20, 0x1805ec0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*TCPListener).accept(0xc42000f038, 0x458e40, 0xc4210a1e90, 0xc4210a1e98) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*TCPListener).Accept(0xc42000f038, 0x1c65af8, 0xc42080fea0, 0x1d71900, 0xc420b1a150) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/http.(*Server).Serve(0xc42106e000, 0x1d6fc00, 0xc42000f038, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/http.Serve(0x1d6fc00, 0xc42000f038, 0x1d5df00, 0xc420b1a090, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420c8e500, 0x1d6fc00, 0xc42000f038, 0xc42079e8d0, 0xc421440300) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 152 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc4204e6200, 0xc42079e8f0, 0xc420e30480) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 153 [sleep]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: time.Sleep(0x22084d89e) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc420234a10, 0xc4204e6200) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 220 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422d02420, 0xc421324d10, 0x30, 0xf9866a) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422d02420, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42020d368, 0xc4207f8e60) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42020d368, 0xc4207f8e60) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 159 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4204fc210, 0xc4210a3d10, 0x30, 0xf9866a) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4204fc210, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc420202618, 0xc4210da000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc420202618, 0xc4210da000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 158 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420243550, 0xc4205c5d10, 0x30, 0xf9866a) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420243550, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42056c7f0, 0xc4201fe870) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42056c7f0, 0xc4201fe870) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 157 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420b54300, 0xc420c73000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).fill(0xc4211a02a0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).ReadByte(0xc4211a02a0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420e10000, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420730080) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4211a0360, 0x16e4b60, 0xc4207300e0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4211a0360, 0x16e4b60, 0xc4207300e0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42138a000, 0x16e4b60, 0xc4207300e0, 0xc4207300e0, 0xc42100e578) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42138a000, 0xc4207300e0, 0x2d, 0xc420e0ec40) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequestHeader(0xc42100e550, 0x1d72340, 0xc42138a000, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4201ff310, 0x8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequest(0xc42100e550, 0x1d72340, 0xc42138a000, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).ServeRequest(0xc42100e550, 0x1d72340, 0xc42138a000, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d0e0, 0xc420b54300, 0xc42100e550) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 197 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421ec8000, 0xc420e88d10, 0x30, 0xf9866a) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421ec8000, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc420203ff0, 0xc422a37040) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc420203ff0, 0xc422a37040) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 166 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753950, 0x72, 0xc420066ce8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc42099e118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc42099e118, 0xc42091b000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc42099e100, 0xc42091b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc42099e100, 0xc42091b000, 0x1000, 0x1000, 0xc420066dd8, 0x1, 0xc420066e08) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc42020c030, 0xc42091b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc420b032c0, 0xc420193540, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc420b032c0, 0xc420193540, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc420b032c0, 0xc420193540, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202433f0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202433f0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 167 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202433f0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 168 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202433f0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 169 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204fe300, 0xc42073a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).fill(0xc420c9b6e0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).ReadByte(0xc420c9b6e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420c99020, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4203564a0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420c9b740, 0x16e4b60, 0xc420356500) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420c9b740, 0x16e4b60, 0xc420356500, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420b5ac40, 0x16e4b60, 0xc420356500, 0xc420356500, 0xc42100e578) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420b5ac40, 0xc420356500, 0x2d, 0xc420ca56c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequestHeader(0xc42100e550, 0x1d72340, 0xc420b5ac40, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4201ff310, 0x8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequest(0xc42100e550, 0x1d72340, 0xc420b5ac40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).ServeRequest(0xc42100e550, 0x1d72340, 0xc420b5ac40, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d0e0, 0xc4204fe300, 0xc42100e550) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 201 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4222be210) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 129 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200be2c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 178 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420b54240, 0xc420bf7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).fill(0xc420d5f5c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).ReadByte(0xc420d5f5c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420b1b980, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4201a0720) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d5f620, 0x16e4b60, 0xc4201a0780) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d5f620, 0x16e4b60, 0xc4201a0780, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420b42940, 0x16e4b60, 0xc4201a0780, 0xc4201a0780, 0xc420da2578) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420b42940, 0xc4201a0780, 0x2d, 0xc420878e00) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequestHeader(0xc420da2550, 0x1d72340, 0xc420b42940, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4201ff310, 0x8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequest(0xc420da2550, 0x1d72340, 0xc420b42940, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).ServeRequest(0xc420da2550, 0x1d72340, 0xc420b42940, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d0e0, 0xc420b54240, 0xc420da2550) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 171 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420b543c0, 0xc420b32000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).fill(0xc420dc05a0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).ReadByte(0xc420dc05a0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420c99c20, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420356cc0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420dc0600, 0x16e4b60, 0xc420356d20) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420dc0600, 0x16e4b60, 0xc420356d20, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420b5b0c0, 0x16e4b60, 0xc420356d20, 0xc420356d20, 0xc420da2578) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420b5b0c0, 0xc420356d20, 0x2d, 0xc4206541e0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequestHeader(0xc420da2550, 0x1d72340, 0xc420b5b0c0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4201ff310, 0x8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequest(0xc420da2550, 0x1d72340, 0xc420b5b0c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).ServeRequest(0xc420da2550, 0x1d72340, 0xc420b5b0c0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d0e0, 0xc420b543c0, 0xc420da2550) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 293 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc422840000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 188 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422d02370) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 200 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4222be210) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 176 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421ec8000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 65 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420243550) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 194 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420243550) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 185 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4210c2cc0, 0xc4214a9700, 0x20, 0x20, 0x169d4e0, 0xc420533d40) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421397890, 0x1d718c0, 0xc4210c2cc0, 0x0, 0xc4210c2c00) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1b80, 0xc4214a9950, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4201f4360, 0xc420be2f00, 0xc421397710, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: reflect.Value.call(0xc420319140, 0xc420202cc8, 0x13, 0x1bda9a0, 0x4, 0xc4214a9d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: reflect.Value.Call(0xc420319140, 0xc420202cc8, 0x13, 0xc4214a9d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*service).call(0xc420ad9d00, 0xc42100e730, 0xc420665168, 0x0, 0xc4213fdf80, 0xc420706100, 0x19758e0, 0xc420be2f00, 0x16, 0x169e020, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).ServeRequest(0xc42100e730, 0x1d72340, 0xc421392080, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d0e0, 0xc4204fe0c0, 0xc42100e730) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 160 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753610, 0x72, 0xc42132ace8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc421240018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc421240018, 0xc42040a000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc421240000, 0xc42040a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc421240000, 0xc42040a000, 0x1000, 0x1000, 0xc4205b45d8, 0x1, 0xc4205b4608) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc420202618, 0xc42040a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc42146e060, 0xc4208582a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc42146e060, 0xc4208582a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc42146e060, 0xc4208582a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4204fc210, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4204fc210) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 161 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4204fc210) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 210 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4204fc210) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 211 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420298000, 0xc420dc8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).fill(0xc420530300) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).ReadByte(0xc420530300, 0x101000000000016, 0x0, 0x2d) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42105f770, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420838040) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4205303c0, 0x16e4b60, 0xc4208380a0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4205303c0, 0x16e4b60, 0xc4208380a0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42146c440, 0x16e4b60, 0xc4208380a0, 0xc4208380a0, 0xc4210da0c8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42146c440, 0xc4208380a0, 0x2d, 0xc420b4a140) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequestHeader(0xc4210da0a0, 0x1d72340, 0xc42146c440, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4201ff310, 0x8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequest(0xc4210da0a0, 0x1d72340, 0xc42146c440, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).ServeRequest(0xc4210da0a0, 0x1d72340, 0xc42146c440, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d0e0, 0xc420298000, 0xc4210da0a0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 189 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422d02370) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 212 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42174c6e0, 0xc421181d10, 0x30, 0xf9866a) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42174c6e0, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42020c018, 0xc421762e10) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42020c018, 0xc421762e10) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 213 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753470, 0x72, 0xc421329ce8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc420739218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc420739218, 0xc420a6f000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc420739200, 0xc420a6f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc420739200, 0xc420a6f000, 0x1000, 0x1000, 0xc4205b9dd8, 0x1, 0xc4205b9e08) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc42020c018, 0xc420a6f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc4216a2060, 0xc42068c740, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc4216a2060, 0xc42068c740, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc4216a2060, 0xc42068c740, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42174c6e0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42174c6e0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 214 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42174c6e0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 215 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42174c6e0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 216 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204fe180, 0xc420c0a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).fill(0xc420b02600) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).ReadByte(0xc420b02600, 0x101000000000016, 0x0, 0x2d) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42178d8c0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42085fe80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420b026c0, 0x16e4b60, 0xc42085fee0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420b026c0, 0x16e4b60, 0xc42085fee0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420ee2480, 0x16e4b60, 0xc42085fee0, 0xc42085fee0, 0xc421762ed8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420ee2480, 0xc42085fee0, 0x2d, 0xc42061a980) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequestHeader(0xc421762eb0, 0x1d72340, 0xc420ee2480, 0x60, 0x2, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequest(0xc421762eb0, 0x1d72340, 0xc420ee2480, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).ServeRequest(0xc421762eb0, 0x1d72340, 0xc420ee2480, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d0e0, 0xc4204fe180, 0xc421762eb0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 186 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422d02370, 0xc420e89d10, 0x30, 0xf9866a) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422d02370, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42020d010, 0xc4207f8cd0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42020d010, 0xc4207f8cd0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 187 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c737533a0, 0x72, 0xc4210a4ce8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc42146a618, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc42146a618, 0xc420fb5000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc42146a600, 0xc420fb5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc42146a600, 0xc420fb5000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc42020d010, 0xc420fb5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc4210c3b00, 0xc4229349b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc4210c3b00, 0xc4229349b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc4210c3b00, 0xc4229349b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422d02370, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422d02370) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 175 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c737537b0, 0x72, 0xc421325ce8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc421ea5098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc421ea5098, 0xc421b46000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc421ea5080, 0xc421b46000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc421ea5080, 0xc421b46000, 0x1000, 0x1000, 0xc421325dd8, 0x0, 0xc421325e08) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc420203ff0, 0xc421b46000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc421671f20, 0xc421ee2f00, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc421671f20, 0xc421ee2f00, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc421671f20, 0xc421ee2f00, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421ec8000, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421ec8000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 221 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753540, 0x72, 0xc4205c4ce8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc422983c18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc422983c18, 0xc42298b000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc422983c00, 0xc42298b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc422983c00, 0xc42298b000, 0x1000, 0x1000, 0xc4205c4dd8, 0x1, 0xc4205c4e08) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc42020d368, 0xc42298b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc421d10360, 0xc422934c20, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc421d10360, 0xc422934c20, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc421d10360, 0xc422934c20, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422d02420, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422d02420) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 218 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4222be000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 219 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4222be000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 199 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753200, 0x72, 0xc4210a2ce8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc421ea5198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc421ea5198, 0xc4226d4000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc421ea5180, 0xc4226d4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc421ea5180, 0xc4226d4000, 0x1000, 0x1000, 0xc4210a2dd8, 0x0, 0xc4210a2e08) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc420203ff8, 0xc4226d4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc42220c3c0, 0xc4229e6500, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc42220c3c0, 0xc4229e6500, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc42220c3c0, 0xc4229e6500, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4222be210, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4222be210) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 217 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753a20, 0x72, 0xc421326ce8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc4226ad298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc4226ad298, 0xc422478000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc4226ad280, 0xc422478000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc4226ad280, 0xc422478000, 0x1000, 0x1000, 0x1d3, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc42056c710, 0xc422478000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc420087800, 0xc421fb7430, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc420087800, 0xc421fb7430, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc420087800, 0xc421fb7430, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4222be000, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4222be000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 224 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420298600, 0xc422477000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).fill(0xc421fe15c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).ReadByte(0xc421fe15c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4222b2a50, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207076e0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421fe1620, 0x16e4b60, 0xc420707760) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421fe1620, 0x16e4b60, 0xc420707760, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422615a80, 0x16e4b60, 0xc420707760, 0xc420707760, 0xc4207f8f28) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422615a80, 0xc420707760, 0x2d, 0xc42264f960) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequestHeader(0xc4207f8f00, 0x1d72340, 0xc422615a80, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4201ff310, 0x8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).readRequest(0xc4207f8f00, 0x1d72340, 0xc422615a80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).ServeRequest(0xc4207f8f00, 0x1d72340, 0xc422615a80, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d0e0, 0xc420298600, 0xc4207f8f00) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 192 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421ec80b0, 0xc420e82d10, 0x30, 0xf9866a) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421ec80b0, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42056c720, 0xc42199a870) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc42056c720, 0xc42199a870) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 193 [IO wait]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.runtime_pollWait(0x7f4c73753af0, 0x72, 0xc42117bce8) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).wait(0xc4228a4018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*pollDesc).waitRead(0xc4228a4018, 0xc4228e1000, 0x1000, 0x1000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: internal/poll.(*FD).Read(0xc4228a4000, 0xc4228e1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*netFD).Read(0xc4228a4000, 0xc4228e1000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net.(*conn).Read(0xc42056c720, 0xc4228e1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: bufio.(*Reader).Read(0xc4222c4780, 0xc422934d30, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadAtLeast(0x1d59aa0, 0xc4222c4780, 0xc422934d30, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: io.ReadFull(0x1d59aa0, 0xc4222c4780, 0xc422934d30, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421ec80b0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421ec80b0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 242 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421ec80b0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 243 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421ec80b0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 244 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4227fc300, 0xc42129b700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4225ec3f0, 0x1d718c0, 0xc4227fc300, 0x0, 0xc4227fc300) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1b80, 0xc42129b950, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4223209c0, 0xc4221f4780, 0xc4225ec270, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: reflect.Value.call(0xc420319140, 0xc42056ca40, 0x13, 0x1bda9a0, 0x4, 0xc42129bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: reflect.Value.Call(0xc420319140, 0xc42056ca40, 0x13, 0xc42129bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*service).call(0xc422bf46c0, 0xc42199a910, 0xc422be3248, 0x0, 0xc422890900, 0xc421ec3a80, 0x19758e0, 0xc4221f4780, 0x16, 0x169e020, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).ServeRequest(0xc42199a910, 0x1d72340, 0xc422bf4700, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d0e0, 0xc420298780, 0xc42199a910) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 225 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4225d7200, 0xc421f0b700, 0x20, 0x20, 0x169d4e0, 0xc42064ea60) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422690ba0, 0x1d718c0, 0xc4225d7200, 0x0, 0xc4225d7200) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1b80, 0xc421f0b950, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421c25aa0, 0xc422492a00, 0xc422690a80, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: reflect.Value.call(0xc420319140, 0xc42020d320, 0x13, 0x1bda9a0, 0x4, 0xc421f0bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: reflect.Value.Call(0xc420319140, 0xc42020d320, 0x13, 0xc421f0bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*service).call(0xc422614a00, 0xc4207f8d70, 0xc422c14718, 0x0, 0xc422983600, 0xc420707fc0, 0x19758e0, 0xc422492a00, 0x16, 0x169e020, ...) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: net/rpc.(*Server).ServeRequest(0xc4207f8d70, 0x1d72340, 0xc4228602c0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d0e0, 0xc422624600, 0xc4207f8d70) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 291 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420296b40) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 232 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b42c0, 0xc420b1c600) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:123 +0x1fd Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc421f1c1c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 233 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b42c0, 0xc420b1c780, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b42c0, 0xc420b1c780) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc421f1c200) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 248 [runnable]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldEnqueue(0xc42088d4a0, 0x1a1c020) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6594 +0x2a0 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4200e1b80, 0xc420533b01, 0xc420533b80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:298 +0x117 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4200e1b80, 0xc421e72de0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4200e1b80, 0xc421e72de0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc4204dc000, 0xc4200e1b80, 0xc421e72de0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 259 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b42c0, 0xc420b1c600, 0xc4228aa780) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc4222e4000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 198 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4222be210, 0xc42117fd10, 0x30, 0xf9866a) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4222be210, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc420c8e440) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc420203ff8, 0xc422a372c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc420c8e440, 0x1d7d320, 0xc420203ff8, 0xc422a372c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 261 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b42c0, 0xc420b1c780, 0xc4228aa900) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc4222e4080) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 209 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4201ff5e0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4200e1b80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 207 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc42029bb80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 208 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc42029bb80) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 250 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc42246c1e0, 0xc4222ae000) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 274 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc422a17880) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 275 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b42c0, 0xc420b1c780, 0x1d6eb80, 0xc422a17880, 0xc421e73320, 0xc421e73380) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc422a178c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 294 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422bf5700, 0xc4210a0ce8, 0x20, 0x20, 0x18, 0xc4206a0160) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42286c120, 0x1d71840, 0xc422bf5700, 0xc4206a0160, 0x184987) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420533b80, 0xc422a79ed0, 0x184987, 0x1d71840, 0xc422bf5700, 0xc420cc50e0, 0xc422691fb0, 0xc420ba0d20, 0x24, 0x2bba400) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420cc5080, 0x1d71840, 0xc422bf5700, 0x184987, 0xc422861f00, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420cc5080, 0x1d71840, 0xc422bf5700) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 295 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc4207f9a40, 0x7f4c687c1218, 0xc422bf5780, 0x1, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:248 +0x2c1 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc4207f9a40, 0x7f4c687c1218, 0xc422bf5780, 0x7f4c687c1218, 0xc422bf5780) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc4204633b0, 0x1d71840, 0xc422bf5780, 0x1, 0x1, 0x1, 0xc42276f7a0, 0xc420e85c78) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:419 +0x7b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc4204633b0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 296 [runnable]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: sync.(*Mutex).Lock(0xc4201ff3b0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/sync/mutex.go:72 +0x2e9 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: log.(*Logger).Output(0xc4201ff3b0, 0x2, 0xc42049b600, 0x7d, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/log/log.go:153 +0x72 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: log.(*Logger).Printf(0xc4201ff3b0, 0x1c429c3, 0x4e, 0xc422a63e48, 0x4, 0x4) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /usr/local/go/src/log/log.go:179 +0x7e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422bf57c0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:197 +0x39d Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 297 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422bf5840) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 251 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc42246c1e0, 0xc4222ae060) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 298 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420ebe2d0, 0x1d71840, 0xc422bf5740) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: goroutine 271 [select]: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc4207f99f0, 0x7f4c687c1218, 0xc422861fc0, 0x1, 0x0, 0x0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:248 +0x2c1 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc4207f99f0, 0x7f4c687c1218, 0xc422861fc0, 0x7f4c687c1218, 0xc422861fc0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsImpl(0xc421074000, 0xc42286c210, 0xc4206a01e0, 0xdcf199c87b8ad6cb, 0xc42109c628, 0xdcf199c87b8ad6cb, 0x1afe340, 0xc42286c220) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:756 +0x89 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).(github.com/hashicorp/nomad/nomad/deploymentwatcher.getAllocsImpl)-fm(0xc42286c210, 0xc4206a01e0, 0xc4201bbaa0, 0xc4211d0818, 0xc4225ede00, 0xc422885000, 0xc4201bbaa0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x3e Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420533b80, 0xc4205c6f20, 0x1, 0x1d71840, 0xc422861fc0, 0xf28cc9, 0xc420cc5080, 0x1d71840, 0xc422bf5700, 0x1) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:149 +0x114 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc421074000, 0x1, 0x0, 0xc, 0xc, 0xc4229e6500, 0x0, 0xc421e72780) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc421074000, 0x1, 0xc4227fdbc0) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25446]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:26:46 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:26:46 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:26:46 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: ==> Starting Nomad agent... Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: ==> Nomad agent configuration: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: Client: false Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: Log Level: INFO Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: Region: amrv01 (DC: test) Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: Server: true Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: Version: 0.8.4 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:46 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:46 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:46 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:46 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:46.971308 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:46 [INFO] serf: Attempting re-join to previously known node: ip-172-25-5-246.amv01: 172.25.5.246:4648 Aug 23 15:26:46 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:46.971383 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.012351 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326462 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326486 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326498 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326509 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [WARN] memberlist: Refuting an alive message Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326755 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326769 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326780 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326836 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326849 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326859 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326919 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.326934 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.327136 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.327153 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.327165 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.327176 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47 [INFO] serf: Re-joined to previously known node: ip-172-25-5-246.amv01: 172.25.5.246:4648 Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.327580 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:26:47 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:47.327600 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 16 Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [INFO] raft: Election won. Tally: 2 Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48.188359 [INFO] nomad: cluster leadership acquired Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:48 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:48 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:49 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:49 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:49 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: 2018/08/23 15:26:49 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 1.207604464s Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: fatal error: concurrent map writes Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 265 [running]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc421f75d08 sp=0xc421f75ce8 pc=0x42d161 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: runtime.mapdelete(0x1844be0, 0xc420308720, 0xc421f75e20) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc421f75d68 sp=0xc421f75d08 pc=0x40a761 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc4202f4480, 0x1d6dd40, 0xc42021b5b0, 0x0, 0xed3109f09) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc421f75e50 sp=0xc421f75d68 pc=0xf2164c Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4204040a0, 0x1d71840, 0xc421a33a80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc421f75fc8 sp=0xc421f75e50 pc=0xf56bb7 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: runtime.goexit() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc421f75fd0 sp=0xc421f75fc8 pc=0x45c711 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 1 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420248770, 0xc4203ee000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420248770, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4202b6000, 0xc4202b6000, 0xc42024afc0, 0x37) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204e1f68) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc42009e058) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: main.main() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 5 [syscall]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: os/signal.signal_recv(0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: os/signal.loop() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by os/signal.init.0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 132 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc4210dcfc0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 23 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4203203c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 180 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420355130, 0xc420e11d10, 0x30, 0xf9866a) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420355130, 0x1c61988, 0xc4202b2000, 0x1d71840, 0xc42112f100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42021a9f8, 0xc420996c80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42021a9f8, 0xc420996c80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 22 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420356300) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 68 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4202bcf00, 0x1d71840, 0xc420441040) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 21 [sleep]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: time.Sleep(0x3b9aca00) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420354000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 20 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc420431840) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 57 [select, locked to thread]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: runtime.selectgo(0xc420050750, 0xc42009e780) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: runtime.ensureSigM.func1() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: runtime.goexit() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 25 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4204249c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc420308b70, 0xc4200695d0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 26 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc42030b590, 0x8, 0xa00000000, 0xc, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc420354160) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 138 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420359b30, 0xc420ef8780) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 79 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584950, 0x72, 0xc42109dce8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc420d9fb18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc420d9fb18, 0xc4212d4000, 0x1000, 0x1000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Read(0xc420d9fb00, 0xc4212d4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).Read(0xc420d9fb00, 0xc4212d4000, 0x1000, 0x1000, 0x8e, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*conn).Read(0xc42000f3f0, 0xc4212d4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).Read(0xc420fb5740, 0xc4207e08d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadAtLeast(0x1d59aa0, 0xc420fb5740, 0xc4207e08d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadFull(0x1d59aa0, 0xc420fb5740, 0xc4207e08d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4212a60b0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4212a60b0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 137 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420359b30, 0x12a05f200, 0xc420e242a0, 0xc420ef8780, 0xc42066ed10) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:110 +0xef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 130 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200c3b80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c3b80, 0xc42066e6a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 136 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420359b30) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 263 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420ea4b40) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 80 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4212a60b0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:49 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 131 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200c3b80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c3b80, 0xc42066e6c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 66 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584e30, 0x72, 0xc4200669a8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc42038f518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc42038f518, 0xc420528000, 0x1000, 0x1000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Read(0xc42038f500, 0xc420528000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).Read(0xc42038f500, 0xc420528000, 0x1000, 0x1000, 0x458600, 0xc420594180, 0x4) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*conn).Read(0xc420568048, 0xc420528000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/http.(*persistConn).Read(0xc4203605a0, 0xc420528000, 0x1000, 0x1000, 0xc420066b98, 0x405675, 0xc4205826c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:26:49 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc420542ae0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).Peek(0xc420542ae0, 0x1, 0x0, 0x0, 0x1, 0xc420dd7620, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/http.(*persistConn).readLoop(0xc4203605a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by net/http.(*Transport).dialConn Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 67 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/http.(*persistConn).writeLoop(0xc4203605a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by net/http.(*Transport).dialConn Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 135 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584c90, 0x72, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc42038f418, 0x72, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc42038f418, 0xc4220e8000, 0x10000, 0x10000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).ReadFrom(0xc42038f400, 0xc4220e8000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).readFrom(0xc42038f400, 0xc4220e8000, 0x10000, 0x10000, 0xeb, 0xeb, 0x1d62b40, 0x412539, 0xc42109ae40) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*UDPConn).readFrom(0xc42020c1e8, 0xc4220e8000, 0x10000, 0x10000, 0x101, 0xc42109ae18, 0xc42109ae90, 0x443cf7) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*UDPConn).ReadFrom(0xc42020c1e8, 0xc4220e8000, 0x10000, 0x10000, 0x10000, 0x10000, 0xa7b4, 0x897246b8, 0x2b96de0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420359b30) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:49 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 74 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584a20, 0x72, 0xc4203d19a8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc420bf8218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc420bf8218, 0xc4210b0000, 0x1000, 0x1000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Read(0xc420bf8200, 0xc4210b0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).Read(0xc420bf8200, 0xc4210b0000, 0x1000, 0x1000, 0x458600, 0xc42017cc00, 0x4) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*conn).Read(0xc42000e008, 0xc4210b0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/http.(*persistConn).Read(0xc4210ae000, 0xc4210b0000, 0x1000, 0x1000, 0xc4203d1b98, 0x405675, 0xc420f88240) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc420fb4fc0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).Peek(0xc420fb4fc0, 0x1, 0x0, 0x0, 0x1, 0xc420f91380, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/http.(*persistConn).readLoop(0xc4210ae000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by net/http.(*Transport).dialConn Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 75 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/http.(*persistConn).writeLoop(0xc4210ae000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by net/http.(*Transport).dialConn Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 264 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc42030aff0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4202b2000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 113 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200c3b80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200c3b80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200c3b80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c3b80, 0xc42066e690) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 178 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202c3550, 0xc420576d10, 0x30, 0xf9866a) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202c3550, 0x1c61988, 0xc4202b2000, 0x1d71840, 0xc42112f100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc420568058, 0xc420c22780) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc420568058, 0xc420c22780) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 134 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584d60, 0x72, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc42038f398, 0x72, 0xc420b46100, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc42038f398, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Accept(0xc42038f380, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).accept(0xc42038f380, 0x0, 0x0, 0xc420306330) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*TCPListener).accept(0xc42020c1d0, 0xc4204b8fc4, 0xc4204b8fc0, 0xc4204b8f30) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*TCPListener).AcceptTCP(0xc42020c1d0, 0x0, 0xc420515020, 0x71a2da) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420359b30) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 133 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420ed2270) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 139 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420359b30, 0x1dcd6500, 0xc420e24300, 0xc420ef8780, 0xc42066ed20) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 140 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc42052f200) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 141 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc42052f200) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 142 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42052f200, 0x1bdd09a, 0x6, 0xc4210dd110) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 143 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42052f200, 0x1bdba6c, 0x5, 0xc4210dd140) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 144 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42052f200, 0x1bdbc6a, 0x5, 0xc4210dd170) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 146 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e24360, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e24360, 0x1dcd6500, 0x7f3d4e4fc1e0, 0xc42020a9c0, 0x0, 0x0, 0xc420542d20) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e24360) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 147 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e243c0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e243c0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e243c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 148 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e24420, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e24420, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e24420) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 149 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e24480, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e24480, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e24480) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 150 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e5847b0, 0x77, 0xc420594180) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc421edc118, 0x77, 0xc4210a9000, 0xc4210a9028, 0x436544) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitWrite(0xc421edc118, 0xc4210a9000, 0x28, 0x611b26) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).WaitWrite(0xc421edc100, 0x1c65988, 0x1d718c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:440 +0x37 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).connect(0xc421edc100, 0x1d718c0, 0xc422775140, 0x0, 0x0, 0x1d5c800, 0xc422570960, 0x0, 0x0, 0x0, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:152 +0x299 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).dial(0xc421edc100, 0x1d718c0, 0xc422775140, 0x1d7a200, 0x0, 0x1d7a200, 0xc422ad1da0, 0xc4210a9248, 0x62f2be) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/sock_posix.go:142 +0xe9 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.socket(0x1d718c0, 0xc422775140, 0x1bda392, 0x3, 0x2, 0x1, 0x0, 0x0, 0x1d7a200, 0x0, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/sock_posix.go:93 +0x1a6 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.internetSocket(0x1d718c0, 0xc422775140, 0x1bda392, 0x3, 0x1d7a200, 0x0, 0x1d7a200, 0xc422ad1da0, 0x1, 0x0, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/ipsock_posix.go:141 +0x129 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.doDialTCP(0x1d718c0, 0xc422775140, 0x1bda392, 0x3, 0x0, 0xc422ad1da0, 0x2bb8aa0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/tcpsock_posix.go:62 +0xb9 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.dialTCP(0x1d718c0, 0xc422775140, 0x1bda392, 0x3, 0x0, 0xc422ad1da0, 0x78fcb0b85, 0x2b96de0, 0x6fc2386b5) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/tcpsock_posix.go:58 +0xe4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.dialSingle(0x1d718c0, 0xc422775140, 0xc421edc080, 0x1d62b00, 0xc422ad1da0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/dial.go:547 +0x375 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.dialSerial(0x1d718c0, 0xc422775140, 0xc421edc080, 0xc421c109c0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/dial.go:515 +0x22d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*Dialer).DialContext(0xc4210a9848, 0x1d71880, 0xc42003e028, 0x1bda392, 0x3, 0xc422570940, 0x11, 0x0, 0x0, 0x0, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/dial.go:397 +0x678 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*Dialer).Dial(0xc4210a9848, 0x1bda392, 0x3, 0xc422570940, 0x11, 0x20, 0x20, 0x17503e0, 0x2ade3c8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/dial.go:320 +0x75 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc420359b30, 0xc422acf3b0, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:758 +0x14e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc420359b30, 0xc422acf3b0, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc420359b30, 0xc420176e00, 0x13, 0x20, 0x3, 0x3, 0xc4206b9cc0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc42052f200, 0xc420176e00, 0x13, 0x20, 0xc4206b9c01, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc4202b2000, 0xc420176e00, 0x13, 0x20, 0xc4206b9cc0, 0x11, 0x10) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc421085f40, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420b69170, 0xc4202b2000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 151 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4202b2000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 152 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4202b2000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 153 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584f00, 0x72, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc42038f298, 0x72, 0xc420b46100, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc42038f298, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Accept(0xc42038f280, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).accept(0xc42038f280, 0x1, 0x1, 0xc4203d3f20) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*TCPListener).accept(0xc42000f3d8, 0x406b4a, 0xc420f900c0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*TCPListener).Accept(0xc42000f3d8, 0xc420f900c0, 0x0, 0x3f800000, 0xc42112f100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4202b2000, 0x1d71840, 0xc42112f100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 154 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4204040a0, 0x3b9aca00, 0xc42045a420) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 155 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc42030aff0, 0x3b9aca00, 0xc42045a420) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 156 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc42035a120, 0x3b9aca00, 0xc42045a420) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 157 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4204249c0, 0x3b9aca00, 0xc42045a420) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 158 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4202b2000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 159 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584af0, 0x72, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc420142a98, 0x72, 0xc420cec000, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc420142a98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Accept(0xc420142a80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).accept(0xc420142a80, 0x412d88, 0x30, 0x1a1fac0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*TCPListener).accept(0xc42020c4d8, 0x5fea4a, 0x1a1fac0, 0xc420598270) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*TCPListener).Accept(0xc42020c4d8, 0xc42003e028, 0x1830320, 0x2ab0680, 0x1b7b4c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/http.(*Server).Serve(0xc4206ba1a0, 0x1d6fc00, 0xc42020c4d8, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/http.Serve(0x1d6fc00, 0xc42020c4d8, 0x1d5df00, 0xc4205981b0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc42112f1c0, 0x1d6fc00, 0xc42020c4d8, 0xc42066f2f0, 0xc420b69230) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 160 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420142b80, 0xc42066f300, 0xc420ef8c60) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 161 [sleep]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: time.Sleep(0x5053b1536) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc420248770, 0xc420142b80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 78 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4212a60b0, 0xc4212d2d10, 0x30, 0xf9866a) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4212a60b0, 0x1c61988, 0xc4202b2000, 0x1d71840, 0xc42112f100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42000f3f0, 0xc420f68780) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42000f3f0, 0xc420f68780) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 81 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4212a60b0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 162 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42043e300, 0xc4212d5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc4212d9aa0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).ReadByte(0xc4212d9aa0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4212dc630, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206dab40) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4212d9b00, 0x16e4b60, 0xc4206daba0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4212d9b00, 0x16e4b60, 0xc4206daba0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42108aa40, 0x16e4b60, 0xc4206daba0, 0xc4206daba0, 0xc420f68898) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42108aa40, 0xc4206daba0, 0x2d, 0xc422cdb120) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequestHeader(0xc420f68870, 0x1d72340, 0xc42108aa40, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420207770, 0x8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequest(0xc420f68870, 0x1d72340, 0xc42108aa40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc420f68870, 0x1d72340, 0xc42108aa40, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc42043e300, 0xc420f68870) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 194 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584880, 0x72, 0xc4212cece8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc420bf8718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc420bf8718, 0xc4212e0000, 0x1000, 0x1000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Read(0xc420bf8700, 0xc4212e0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).Read(0xc420bf8700, 0xc4212e0000, 0x1000, 0x1000, 0xc421083dd8, 0x1, 0xc421083e08) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*conn).Read(0xc420568058, 0xc4212e0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).Read(0xc420f676e0, 0xc420788810, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadAtLeast(0x1d59aa0, 0xc420f676e0, 0xc420788810, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadFull(0x1d59aa0, 0xc420f676e0, 0xc420788810, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202c3550, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202c3550) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 195 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202c3550) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 196 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202c3550) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 197 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42032e480, 0xc4212fe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc4212fc900) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).ReadByte(0xc4212fc900, 0x101000000000016, 0x0, 0x2d) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4212fa720, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207036e0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4212fc960, 0x16e4b60, 0xc420703760) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4212fc960, 0x16e4b60, 0xc420703760, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42112f640, 0x16e4b60, 0xc420703760, 0xc420703760, 0xc4210d6d48) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42112f640, 0xc420703760, 0x2d, 0xc421ee0020) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequestHeader(0xc4210d6d20, 0x1d72340, 0xc42112f640, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequest(0xc4210d6d20, 0x1d72340, 0xc42112f640, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc4210d6d20, 0x1d72340, 0xc42112f640, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc42032e480, 0xc4210d6d20) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 163 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42032e540, 0xc42024e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc4209947e0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).ReadByte(0xc4209947e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420c387e0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206c6300) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420994840, 0x16e4b60, 0xc4206c6360) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420994840, 0x16e4b60, 0xc4206c6360, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420d95200, 0x16e4b60, 0xc4206c6360, 0xc4206c6360, 0xc420f68898) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420d95200, 0xc4206c6360, 0x2d, 0xc422cdb6c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequestHeader(0xc420f68870, 0x1d72340, 0xc420d95200, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420207770, 0x8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequest(0xc420f68870, 0x1d72340, 0xc420d95200, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc420f68870, 0x1d72340, 0xc420d95200, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc42032e540, 0xc420f68870) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 177 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4203540b0, 0xc4212d1d10, 0x30, 0xf9866a) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4203540b0, 0x1c61988, 0xc4202b2000, 0x1d71840, 0xc42112f100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42020c030, 0xc421428a00) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42020c030, 0xc421428a00) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 164 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202c34a0, 0xc421097d10, 0x30, 0xf9866a) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202c34a0, 0x1c61988, 0xc4202b2000, 0x1d71840, 0xc42112f100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42020c248, 0xc420996370) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42020c248, 0xc420996370) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 165 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584bc0, 0x72, 0xc4212cdce8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc420bf8118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc420bf8118, 0xc420ef0000, 0x1000, 0x1000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Read(0xc420bf8100, 0xc420ef0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).Read(0xc420bf8100, 0xc420ef0000, 0x1000, 0x1000, 0xac, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*conn).Read(0xc42020c248, 0xc420ef0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).Read(0xc420994c00, 0xc42067eba0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadAtLeast(0x1d59aa0, 0xc420994c00, 0xc42067eba0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadFull(0x1d59aa0, 0xc420994c00, 0xc42067eba0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202c34a0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202c34a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 166 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202c34a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 167 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202c34a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 168 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421ff0240, 0xc4212f7700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4223469c0, 0x1d718c0, 0xc421ff0240, 0x0, 0xc421ff0200) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202b2000, 0xc4212f7950, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420087740, 0xc421dec080, 0xc4223468a0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: reflect.Value.call(0xc420454900, 0xc42000fce8, 0x13, 0x1bda9a0, 0x4, 0xc4212ccd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: reflect.Value.Call(0xc420454900, 0xc42000fce8, 0x13, 0xc4212ccd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*service).call(0xc420b47480, 0xc420996410, 0xc422318168, 0x0, 0xc420a3b880, 0xc4206860c0, 0x19758e0, 0xc421dec080, 0x16, 0x169e020, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc420996410, 0x1d72340, 0xc420b474c0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc42043e480, 0xc420996410) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 258 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4217320b0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 209 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4217320b0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 219 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420c860c0, 0xc420679000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc420455800) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).ReadByte(0xc420455800, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421b34870, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42071c620) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420455860, 0x16e4b60, 0xc42071c680) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420455860, 0x16e4b60, 0xc42071c680, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421eb8080, 0x16e4b60, 0xc42071c680, 0xc42071c680, 0xc420996d48) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421eb8080, 0xc42071c680, 0x2d, 0xc422cdb260) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequestHeader(0xc420996d20, 0x1d72340, 0xc421eb8080, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420207770, 0x8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequest(0xc420996d20, 0x1d72340, 0xc421eb8080, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc420996d20, 0x1d72340, 0xc421eb8080, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc420c860c0, 0xc420996d20) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 181 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e5846e0, 0x72, 0xc420edace8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc420ec5e18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc420ec5e18, 0xc420726000, 0x1000, 0x1000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Read(0xc420ec5e00, 0xc420726000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).Read(0xc420ec5e00, 0xc420726000, 0x1000, 0x1000, 0xc420edadd8, 0x1, 0xc420edae08) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*conn).Read(0xc42021a9f8, 0xc420726000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).Read(0xc420189aa0, 0xc4200ec540, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadAtLeast(0x1d59aa0, 0xc420189aa0, 0xc4200ec540, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadFull(0x1d59aa0, 0xc420189aa0, 0xc4200ec540, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420355130, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420355130) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 182 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420355130) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 183 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420355130) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 184 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420c87440, 0xc420727000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc4212fd500) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).ReadByte(0xc4212fd500, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4203948d0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420681a60) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4212fd560, 0x16e4b60, 0xc420681ac0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4212fd560, 0x16e4b60, 0xc420681ac0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420bad140, 0x16e4b60, 0xc420681ac0, 0xc420681ac0, 0xc420996d48) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420bad140, 0xc420681ac0, 0x2d, 0xc422cdb020) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequestHeader(0xc420996d20, 0x1d72340, 0xc420bad140, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420207770, 0x8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequest(0xc420996d20, 0x1d72340, 0xc420bad140, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc420996d20, 0x1d72340, 0xc420bad140, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc420c87440, 0xc420996d20) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 187 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420bad3c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 212 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4200c3b80, 0xc42043e780, 0x185fbd, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:171 +0x7a2 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200c3b80, 0xc42043e780) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:135 +0x258 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c3b80, 0xc4206606e0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 200 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200c3b80, 0xc42043e780, 0xc420d9b0e0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:351 +0x4fe Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c3b80, 0xc4206c6820) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 213 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200c3b80, 0xc42043e840, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200c3b80, 0xc42043e840) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c3b80, 0xc420660700) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 185 [runnable]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldBlock(0xc4216169a0, 0x1a1c000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6607 +0x2a4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4202b2000, 0xc420567a01, 0xc420567a40) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:300 +0x152 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4202b2000, 0xc420e59f20, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4202b2000, 0xc420e59f20) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420764160, 0xc4202b2000, 0xc420e59f20) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 186 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200c3b80, 0xc42043e840, 0xc420d64000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c3b80, 0xc420681c60) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 188 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200c3b80, 0xc42043e840, 0x1d6eb80, 0xc420bad3c0, 0xc420d65140, 0xc420d65200) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c3b80, 0xc420bad400) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 208 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584200, 0x72, 0xc420e0dce8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc421204218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc421204218, 0xc420b64000, 0x1000, 0x1000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Read(0xc421204200, 0xc420b64000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).Read(0xc421204200, 0xc420b64000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*conn).Read(0xc42021a1a8, 0xc420b64000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).Read(0xc4204f2ba0, 0xc420b35c40, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadAtLeast(0x1d59aa0, 0xc4204f2ba0, 0xc420b35c40, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadFull(0x1d59aa0, 0xc4204f2ba0, 0xc420b35c40, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4217320b0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4217320b0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 259 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421ff0180, 0xc420f95700, 0x20, 0x20, 0x169d4e0, 0xc4203221a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422346570, 0x1d718c0, 0xc421ff0180, 0x0, 0xc421ff0100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202b2000, 0xc420f95950, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420ed0f00, 0xc421dec000, 0xc422346450, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: reflect.Value.call(0xc420454900, 0xc42021a838, 0x13, 0x1bda9a0, 0x4, 0xc420ce5d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: reflect.Value.Call(0xc420454900, 0xc42021a838, 0x13, 0xc420ce5d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*service).call(0xc421ed1540, 0xc42030afa0, 0xc4223180f0, 0x0, 0xc42064f200, 0xc4218d5080, 0x19758e0, 0xc421dec000, 0x16, 0x169e020, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc42030afa0, 0x1d72340, 0xc421ed1600, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc420c86180, 0xc42030afa0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 214 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202c3290, 0xc420e0ad10, 0x30, 0xf9866a) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202c3290, 0x1c61988, 0xc4202b2000, 0x1d71840, 0xc42112f100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42000ffe8, 0xc4210880a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42000ffe8, 0xc4210880a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 189 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200cc2c0, 0xc420ed9d10, 0x30, 0xf9866a) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200cc2c0, 0x1c61988, 0xc4202b2000, 0x1d71840, 0xc42112f100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42000e520, 0xc421089a40) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42000e520, 0xc421089a40) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 190 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584540, 0x72, 0xc420ce9ce8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc420818d18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc420818d18, 0xc4212e1000, 0x1000, 0x1000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Read(0xc420818d00, 0xc4212e1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).Read(0xc420818d00, 0xc4212e1000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*conn).Read(0xc42000e520, 0xc4212e1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).Read(0xc420086f60, 0xc4207c4000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadAtLeast(0x1d59aa0, 0xc420086f60, 0xc4207c4000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadFull(0x1d59aa0, 0xc420086f60, 0xc4207c4000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200cc2c0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200cc2c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 191 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200cc2c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 192 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200cc2c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 193 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421086420, 0xc4210ab700, 0x20, 0x20, 0x169d4e0, 0xc4203221a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4203284e0, 0x1d718c0, 0xc421086420, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202b2000, 0xc4210ab950, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420f671a0, 0xc420ff9d80, 0xc4209ca2d0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: reflect.Value.call(0xc420454900, 0xc42000ff88, 0x13, 0x1bda9a0, 0x4, 0xc4210abd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: reflect.Value.Call(0xc420454900, 0xc42000ff88, 0x13, 0xc4210abd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*service).call(0xc420cece00, 0xc421089ae0, 0xc420788258, 0x0, 0xc420ec5780, 0xc4207e9820, 0x19758e0, 0xc420ff9d80, 0x16, 0x169e020, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc421089ae0, 0x1d72340, 0xc420cece40, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc420c86000, 0xc421089ae0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 206 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42032e000, 0xc420883000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc4219e8ea0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).ReadByte(0xc4219e8ea0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4219fb800, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42107c920) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4219e8f00, 0x16e4b60, 0xc42107c9a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4219e8f00, 0x16e4b60, 0xc42107c9a0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421876bc0, 0x16e4b60, 0xc42107c9a0, 0xc42107c9a0, 0xc421428ac8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421876bc0, 0xc42107c9a0, 0x2d, 0xc421ee0100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequestHeader(0xc421428aa0, 0x1d72340, 0xc421876bc0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420207770, 0x8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequest(0xc421428aa0, 0x1d72340, 0xc421876bc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc421428aa0, 0x1d72340, 0xc421876bc0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc42032e000, 0xc421428aa0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 215 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e584470, 0x72, 0xc420062ce8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc420c26018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc420c26018, 0xc420675000, 0x1000, 0x1000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Read(0xc420c26000, 0xc420675000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).Read(0xc420c26000, 0xc420675000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*conn).Read(0xc42000ffe8, 0xc420675000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).Read(0xc42120a060, 0xc4200ecf40, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadAtLeast(0x1d59aa0, 0xc42120a060, 0xc4200ecf40, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadFull(0x1d59aa0, 0xc42120a060, 0xc4200ecf40, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202c3290, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202c3290) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 216 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202c3290) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 217 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202c3290) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 218 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4223f6540, 0xc420f99700, 0x20, 0x20, 0x169d4e0, 0xc4203221a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4227c8bd0, 0x1d718c0, 0xc4223f6540, 0x0, 0xc4223f6500) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202b2000, 0xc420f99950, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42120bec0, 0xc421238200, 0xc4227c8ab0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: reflect.Value.call(0xc420454900, 0xc42020cbc0, 0x13, 0x1bda9a0, 0x4, 0xc420e0bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: reflect.Value.Call(0xc420454900, 0xc42020cbc0, 0x13, 0xc420e0bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*service).call(0xc420c96400, 0xc421088140, 0xc422318218, 0x0, 0xc421384900, 0xc42020e180, 0x19758e0, 0xc421238200, 0x16, 0x169e020, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc421088140, 0x1d72340, 0xc420c96440, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc4203560c0, 0xc421088140) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 242 [IO wait]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.runtime_pollWait(0x7f3d4e5842d0, 0x72, 0xc420edfce8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).wait(0xc421204118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*pollDesc).waitRead(0xc421204118, 0xc420b4c000, 0x1000, 0x1000) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: internal/poll.(*FD).Read(0xc421204100, 0xc420b4c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).Read(0xc421204100, 0xc420b4c000, 0x1000, 0x1000, 0xc4204b3dd8, 0x1, 0xc4204b3e08) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*conn).Read(0xc42020c030, 0xc420b4c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).Read(0xc4218c3800, 0xc42169c6f0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadAtLeast(0x1d59aa0, 0xc4218c3800, 0xc42169c6f0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: io.ReadFull(0x1d59aa0, 0xc4218c3800, 0xc42169c6f0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4203540b0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4203540b0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 243 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4203540b0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 244 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4203540b0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 231 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42043e3c0, 0xc421623000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc4223f62a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).ReadByte(0xc4223f62a0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4227c8390, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4227d21a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4223f6300, 0x16e4b60, 0xc4227d2200) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4223f6300, 0x16e4b60, 0xc4227d2200, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4228e2200, 0x16e4b60, 0xc4227d2200, 0xc4227d2200, 0xc420996438) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4228e2200, 0xc4227d2200, 0x2d, 0xc421ee0360) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequestHeader(0xc420996410, 0x1d72340, 0xc4228e2200, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420207770, 0x8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequest(0xc420996410, 0x1d72340, 0xc4228e2200, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc420996410, 0x1d72340, 0xc4228e2200, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc42043e3c0, 0xc420996410) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 207 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4217320b0, 0xc420ce4d10, 0x30, 0xf9866a) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4217320b0, 0x1c61988, 0xc4202b2000, 0x1d71840, 0xc42112f100) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42021a1a8, 0xc42030aeb0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d320, 0xc42021a1a8, 0xc42030aeb0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 246 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420356000, 0xc42091e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc4210da7e0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).ReadByte(0xc4210da7e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420f63080, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420686680) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210da840, 0x16e4b60, 0xc420686700) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210da840, 0x16e4b60, 0xc420686700, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420c65980, 0x16e4b60, 0xc420686700, 0xc420686700, 0xc420f68898) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420c65980, 0xc420686700, 0x2d, 0xc421064a00) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequestHeader(0xc420f68870, 0x1d72340, 0xc420c65980, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420207770, 0x8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequest(0xc420f68870, 0x1d72340, 0xc420c65980, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc420f68870, 0x1d72340, 0xc420c65980, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc420356000, 0xc420f68870) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 247 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net.(*netFD).connect.func2(0x1d718c0, 0xc422775140, 0xc421edc100, 0xc421f900c0, 0xc421f90060) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:129 +0xde Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by net.(*netFD).connect Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/fd_unix.go:128 +0x286 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 262 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420ea4b40) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 248 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420356180, 0xc42256a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc4228166c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).ReadByte(0xc4228166c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422849b90, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4225383a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422816720, 0x16e4b60, 0xc422538400) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422816720, 0x16e4b60, 0xc422538400, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422302b80, 0x16e4b60, 0xc422538400, 0xc422538400, 0xc42030afc8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422302b80, 0xc422538400, 0x2d, 0xc422cdb480) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequestHeader(0xc42030afa0, 0x1d72340, 0xc422302b80, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420207770, 0x8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequest(0xc42030afa0, 0x1d72340, 0xc422302b80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc42030afa0, 0x1d72340, 0xc422302b80, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc420356180, 0xc42030afa0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 249 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4203563c0, 0xc421368000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).fill(0xc422816b40) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: bufio.(*Reader).ReadByte(0xc422816b40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42243a2d0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4225385c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422816ba0, 0x16e4b60, 0xc422538620) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422816ba0, 0x16e4b60, 0xc422538620, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422302e80, 0x16e4b60, 0xc422538620, 0xc422538620, 0xc421088168) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422302e80, 0xc422538620, 0x2d, 0xc421be0ac0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequestHeader(0xc421088140, 0x1d72340, 0xc422302e80, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420207770, 0x8) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).readRequest(0xc421088140, 0x1d72340, 0xc422302e80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: net/rpc.(*Server).ServeRequest(0xc421088140, 0x1d72340, 0xc422302e80, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202b2000, 0x1d71840, 0xc42112f100, 0x1d7d0e0, 0xc4203563c0, 0xc421088140) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 266 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc42035a120) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 267 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc42035a120) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 268 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc421ae88a0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 269 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc421a33b80, 0xc421096ce8, 0x20, 0x20, 0x18, 0xc4225389c0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42243ab40, 0x1d71840, 0xc421a33b80, 0xc4225389c0, 0x184987) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420567a40, 0xc422a01ed0, 0x184987, 0x1d71840, 0xc421a33b80, 0xc42038f9e0, 0xc421ae8870, 0xc420eb45d0, 0x24, 0x2bba400) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc42038f980, 0x1d71840, 0xc421a33b80, 0x184987, 0xc422303280, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc42038f980, 0x1d71840, 0xc421a33b80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 270 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc421a33e40, 0xc422a05770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42243ad50, 0x1d71840, 0xc421a33e40, 0x0, 0x1849b5) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420567a40, 0xc422a05960, 0x1849b5, 0x1d71840, 0xc421a33e40, 0x552bde, 0xc422a05950, 0xc422a05970, 0xf33ba5, 0xc42047af38) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc42047aee0, 0x1d71840, 0xc421a33e40, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc42047aee0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 271 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc421a33d80, 0xc422aa3a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421ad6000, 0x1d71840, 0xc421a33d80, 0xc421ad6180, 0x1833a4) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420567a40, 0xc422aa3c68, 0x1833a4, 0x1d71840, 0xc421a33d80, 0x552bde, 0xc42030ad70, 0x2, 0xc420e74500, 0x40) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc421a33f00, 0x1833a4, 0x3b, 0xc421f77de8, 0x1, 0x1) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc421a33f00) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 272 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc421c2c080) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 221 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4223472c0, 0xc4227dc720) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 222 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4223472c0, 0xc4227dc7e0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 273 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420359d10, 0x1d71840, 0xc421a33d80) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 250 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc420ef2960) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: goroutine 232 [select]: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422303340, 0xc420e0bd28, 0x20, 0x20, 0xc42002cc00, 0x1bd9a98) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4227c9260, 0x1d71840, 0xc422303340, 0xc4227d2700, 0x1849b5) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420567a40, 0xc422a07f20, 0x1849b5, 0x1d71840, 0xc422303340, 0xc, 0xc, 0x0, 0x0, 0xc4204b3fa0) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc420ef2960, 0x1849b5, 0x0, 0xc, 0xc, 0xc42169c6f0, 0x0, 0xc4206ac600) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc420ef2960, 0x1849b5, 0xc4223f6900) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25466]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:26:49 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:26:49 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:26:49 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: ==> Starting Nomad agent... Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: ==> Nomad agent configuration: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: Client: false Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: Log Level: INFO Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: Region: amrv01 (DC: test) Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: Server: true Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: Version: 0.8.4 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:49 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:49 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:49 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:49 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:49.964898 [INFO] nomad: starting 4 scheduling worker(s) for [system service batch _core] Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:49 [INFO] serf: Attempting re-join to previously known node: vol-cl-control-01.global: 172.16.9.87:4648 Aug 23 15:26:49 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:49.965164 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.120163 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.120177 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.120188 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.120200 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [WARN] memberlist: Refuting an alive message Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274216 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274239 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274253 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274264 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274275 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274548 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274572 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274585 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274807 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50 [INFO] serf: Re-joined to previously known node: vol-cl-control-01.global: 172.16.9.87:4648 Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274819 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274831 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274842 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274854 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274866 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:26:50 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:50.274878 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:51 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:53 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:53 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55.038703 [ERR] http: Request /v1/agent/health?type=server, error: {"server":{"ok":false,"message":"rpc error: stream closed"}} Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 19 Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [INFO] raft: Election won. Tally: 2 Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55.932680 [INFO] nomad: cluster leadership acquired Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:55 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:55 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:56.047300 [INFO] server.nomad: successfully contacted 19 Nomad Servers Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:56 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:56 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:56 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:56 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:56 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: 2018/08/23 15:26:56 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: fatal error: concurrent map read and map write Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 293 [running]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc422d439c0 sp=0xc422d439a0 pc=0x42d161 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: runtime.mapaccess2(0x1844be0, 0xc4201ab4d0, 0xc422d43a88, 0x412539, 0xc420d14030) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc422d43a08 sp=0xc422d439c0 pc=0x409c75 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc4201be8e0, 0x1d6dd40, 0xc420d14030, 0x2faf6a57, 0xed31087cc, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:94 +0xe2 fp=0xc422d43af8 sp=0xc422d43a08 pc=0xf20ca2 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc42033b040, 0xc42088e840, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc422d43b60 sp=0xc422d43af8 pc=0xf5388e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc42033b040, 0xc42088e840) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc422d43b98 sp=0xc422d43b60 pc=0xf534b3 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4202ae780, 0xc4200eb401, 0xc4200eb4c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc422d43c88 sp=0xc422d43b98 pc=0xf77602 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4202ae780, 0xc422d8d080, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc422d43d38 sp=0xc422d43c88 pc=0xf76fff Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4202ae780, 0xc422d8d080) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc422d43fa0 sp=0xc422d43d38 pc=0xf76b90 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420808070, 0xc4202ae780, 0xc422d8d080) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc422d43fc8 sp=0xc422d43fa0 pc=0xfb8e9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: runtime.goexit() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc422d43fd0 sp=0xc422d43fc8 pc=0x45c711 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 1 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420241260, 0xc420422640) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420241260, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4202b2640, 0xc4202b2640, 0xc420283de0, 0x37) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204d7f68) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc42009e058) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: main.main() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 5 [syscall]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: os/signal.signal_recv(0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: os/signal.loop() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by os/signal.init.0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 225 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421453b80, 0xc422d2cd10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421453b80, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc4204ab050, 0xc420383040) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc4204ab050, 0xc420383040) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 130 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200c2dc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c2dc0, 0xc42072c080) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 281 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200c2dc0, 0xc42265a600, 0xc422d7b440) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c2dc0, 0xc421d95a60) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 59 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc42042ae00, 0x1d71840, 0xc420441700) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 66 [select, locked to thread]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: runtime.selectgo(0xc4204e7750, 0xc42031a420) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: runtime.ensureSigM.func1() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: runtime.goexit() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 67 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42042abc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 68 [sleep]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: time.Sleep(0x3b9aca00) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc42048ec60) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 69 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420488540) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 70 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc42029aa80) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 103 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420476dc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 72 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4203985b0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4201ab920, 0xc42020c450) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 73 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420438e10, 0x1c636a0, 0xc42048edc0, 0x1d71840, 0xc42055d7c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc42048edc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 229 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4218fa0c0, 0xc420e14000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc4213b56e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).ReadByte(0xc4213b56e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4221a7770, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421dc6ae0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4213b5740, 0x16e4b60, 0xc421dc6b40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4213b5740, 0x16e4b60, 0xc421dc6b40, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422c84740, 0x16e4b60, 0xc421dc6b40, 0xc421dc6b40, 0xc4211bf158) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422c84740, 0xc421dc6b40, 0x2d, 0xc422275060) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequestHeader(0xc4211bf130, 0x1d72340, 0xc422c84740, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420438870, 0x8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequest(0xc4211bf130, 0x1d72340, 0xc422c84740, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc4211bf130, 0x1d72340, 0xc422c84740, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc4218fa0c0, 0xc4211bf130) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 102 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420476dc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 101 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160ba20, 0x72, 0xc420061ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc420142598, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc420142598, 0xc4213da000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc420142580, 0xc4213da000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc420142580, 0xc4213da000, 0x1000, 0x1000, 0xc420061dd8, 0x1, 0xc420061e08) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc4204aa038, 0xc4213da000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc420e7e2a0, 0xc420808200, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc420e7e2a0, 0xc420808200, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc420e7e2a0, 0xc420808200, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420476dc0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420476dc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 129 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200c2dc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200c2dc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200c2dc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c2dc0, 0xc42072c070) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 104 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4213fc300, 0xc421429700, 0x20, 0x20, 0x169d4e0, 0xc4216367a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4215bbaa0, 0x1d718c0, 0xc4213fc300, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202ae780, 0xc421429950, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4213fc060, 0xc42130e000, 0xc420eeca80, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.call(0xc420452de0, 0xc4204aa880, 0x13, 0x1bda9a0, 0x4, 0xc421429d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.Call(0xc420452de0, 0xc4204aa880, 0x13, 0xc421429d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*service).call(0xc4213c8440, 0xc420e82280, 0xc420ba05c8, 0x0, 0xc420f83f00, 0xc4207a4160, 0x19758e0, 0xc42130e000, 0x16, 0x169e020, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc420e82280, 0x1d72340, 0xc4213c8480, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc42039e300, 0xc420e82280) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 131 [select]: Aug 23 15:26:56 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200c2dc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c2dc0, 0xc42072c090) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 57 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160be30, 0x72, 0xc4200629a8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc42039a298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc42039a298, 0xc42038c000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc42039a280, 0xc42038c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc42039a280, 0xc42038c000, 0x1000, 0x1000, 0x458600, 0xc420e42a80, 0x4) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc4204aa008, 0xc42038c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.(*persistConn).Read(0xc4202aa6c0, 0xc42038c000, 0x1000, 0x1000, 0xc420062b98, 0x405675, 0xc42031a060) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc420086a20) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Peek(0xc420086a20, 0x1, 0x0, 0x0, 0x1, 0xc420dd8cc0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:26:56 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.(*persistConn).readLoop(0xc4202aa6c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by net/http.(*Transport).dialConn Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 58 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.(*persistConn).writeLoop(0xc4202aa6c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by net/http.(*Transport).dialConn Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 318 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200c2dc0, 0xc42265a6c0, 0x1d6eb80, 0xc4229bf340, 0xc422d6a6c0, 0xc422d6a840) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c2dc0, 0xc4229bf440) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 169 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421c66000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 262 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421452000, 0xc422d2dd10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421452000, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42000f768, 0xc4211fd590) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42000f768, 0xc4211fd590) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 170 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421c66000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 168 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160b6e0, 0x72, 0xc4213e9ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc422da8618, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc422da8618, 0xc422574000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc422da8600, 0xc422574000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc422da8600, 0xc422574000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc4204abc40, 0xc422574000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc422866000, 0xc422398410, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc422866000, 0xc422398410, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc422866000, 0xc422398410, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421c66000, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421c66000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 277 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160b130, 0x72, 0xc422d2ace8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc421fb7398, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc421fb7398, 0xc4226aa000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc421fb7380, 0xc4226aa000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc421fb7380, 0xc4226aa000, 0x1000, 0x1000, 0xc422d2add8, 0x0, 0xc422d2ae08) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42020f4b8, 0xc4226aa000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc420e7f560, 0xc421bef600, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc420e7f560, 0xc421bef600, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc420e7f560, 0xc421bef600, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422321130, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422321130) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 100 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420476dc0, 0xc42057ad10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420476dc0, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc4204aa038, 0xc420e82190) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc4204aa038, 0xc420e82190) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 132 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420d90600) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 133 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc42049e270) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 134 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160bd60, 0x72, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc420e36598, 0x72, 0xc4211c0100, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc420e36598, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Accept(0xc420e36580, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).accept(0xc420e36580, 0xc4204ab3a8, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*TCPListener).accept(0xc42021a500, 0x458e40, 0xc420050f90, 0xc420050f98) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*TCPListener).AcceptTCP(0xc42021a500, 0x1c63658, 0xc420f5d590, 0xc4204ab3a8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420f5d590) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 135 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160bc90, 0x72, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc420e36718, 0x72, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc420e36718, 0xc423232000, 0x10000, 0x10000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).ReadFrom(0xc420e36700, 0xc423232000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).readFrom(0xc420e36700, 0xc423232000, 0x10000, 0x10000, 0x347, 0x347, 0x1d62b40, 0x412539, 0xc4213bce40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*UDPConn).readFrom(0xc42021a508, 0xc423232000, 0x10000, 0x10000, 0x101, 0xc4213bce18, 0xc4213bce90, 0x443cf7) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*UDPConn).ReadFrom(0xc42021a508, 0xc423232000, 0x10000, 0x10000, 0x10000, 0x10000, 0x1ae72, 0x177b015a0, 0x2b96de0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420f5d590) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 136 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420f5d590) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 137 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420f5d590, 0x12a05f200, 0xc420e5d020, 0xc420f78d20, 0xc42072c6a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 138 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420f5d590, 0xc420f78d20) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 139 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420f5d590, 0x1dcd6500, 0xc420e5d080, 0xc420f78d20, 0xc42072c6b0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 140 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc42017f080) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 141 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc42017f080) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 142 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42017f080, 0x1bdd09a, 0x6, 0xc420d90750) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 143 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42017f080, 0x1bdba6c, 0x5, 0xc420d90780) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 144 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42017f080, 0x1bdbc6a, 0x5, 0xc420d907b0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 200 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421124000, 0xc4213c3d10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421124000, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42021a510, 0xc42110c000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42021a510, 0xc42110c000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 146 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e5d0e0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e5d0e0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e5d0e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 147 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e5d140, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e5d140, 0x1dcd6500, 0x7eff615d2d68, 0xc420202140, 0x0, 0x0, 0xc42106e5a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e5d140) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 148 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e5d1a0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e5d1a0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e5d1a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 149 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e5d200, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e5d200, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e5d200) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 150 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420eec7b0, 0xc4202ae780) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 151 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4202ae780) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 152 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4202ae780) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 153 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160bf00, 0x72, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc420552898, 0x72, 0xc4211c0100, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc420552898, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Accept(0xc420552880, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).accept(0xc420552880, 0x1, 0x1, 0xc420576f20) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*TCPListener).accept(0xc42021b1b8, 0x406b4a, 0xc420efc780, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*TCPListener).Accept(0xc42021b1b8, 0xc420efc780, 0x0, 0x3f800000, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 154 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc42033b040, 0x3b9aca00, 0xc42031a8a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 155 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc420438be0, 0x3b9aca00, 0xc42031a8a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 156 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420486510, 0x3b9aca00, 0xc42031a8a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 157 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4203985b0, 0x3b9aca00, 0xc42031a8a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 158 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4202ae780) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 159 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160bbc0, 0x72, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc42096da98, 0x72, 0xc4213ba100, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc42096da98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Accept(0xc42096da80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).accept(0xc42096da80, 0x402037, 0xc4220fe1c0, 0x1805ec0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*TCPListener).accept(0xc42021ae48, 0x458e40, 0xc4213e6e90, 0xc4213e6e98) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*TCPListener).Accept(0xc42021ae48, 0x1c65af8, 0xc4220fe140, 0x1d71900, 0xc420fcc4b0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.(*Server).Serve(0xc420336410, 0x1d6fc00, 0xc42021ae48, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.Serve(0x1d6fc00, 0xc42021ae48, 0x1d5df00, 0xc420fcc3f0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420d87000, 0x1d6fc00, 0xc42021ae48, 0xc42072cbf0, 0xc420eec870) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 160 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc42096db00, 0xc42072cc00, 0xc420f78f60) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 161 [sleep]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: time.Sleep(0x3c2ef6b15) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc420241260, 0xc42096db00) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 187 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421c66000, 0xc4211bbd10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421c66000, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc4204abc40, 0xc42290b680) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc4204abc40, 0xc42290b680) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 188 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200cc2c0, 0xc420a4ad10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200cc2c0, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42020e018, 0xc421de62d0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42020e018, 0xc421de62d0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 178 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160b950, 0x72, 0xc4213e29a8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc420aea098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc420aea098, 0xc4213f4000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc420aea080, 0xc4213f4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc420aea080, 0xc4213f4000, 0x1000, 0x1000, 0x458600, 0xc42017ef00, 0x4) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42021ae88, 0xc4213f4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.(*persistConn).Read(0xc4213e0000, 0xc4213f4000, 0x1000, 0x1000, 0xc4213e2b98, 0x405675, 0xc420f79980) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc420e5d320) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Peek(0xc420e5d320, 0x1, 0x0, 0x0, 0x1, 0xc420f79e00, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.(*persistConn).readLoop(0xc4213e0000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by net/http.(*Transport).dialConn Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 214 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420477e40, 0xc4213bfd10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420477e40, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc4204aa3f0, 0xc4216fe0a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc4204aa3f0, 0xc4216fe0a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 278 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422321130) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 179 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.(*persistConn).writeLoop(0xc4213e0000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by net/http.(*Transport).dialConn Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 49 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160b880, 0x72, 0xc4213e79a8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc42014b698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc42014b698, 0xc4213eb000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc42014b680, 0xc4213eb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc42014b680, 0xc4213eb000, 0x1000, 0x1000, 0xc42003e028, 0x1bda392, 0x3) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42020e0c8, 0xc4213eb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.(*persistConn).Read(0xc4213f8000, 0xc4213eb000, 0x1000, 0x1000, 0x405486, 0xc4213e7b98, 0x42bc79) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc420fcbc20) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Peek(0xc420fcbc20, 0x1, 0xc420eaede0, 0xc42053a480, 0xc420fcb500, 0x60, 0x1b18960) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.(*persistConn).readLoop(0xc4213f8000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by net/http.(*Transport).dialConn Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 194 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/http.(*persistConn).writeLoop(0xc4213f8000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by net/http.(*Transport).dialConn Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 276 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422321130, 0xc422d30d10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422321130, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42020f4b8, 0xc420382aa0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42020f4b8, 0xc420382aa0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 201 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160b540, 0x72, 0xc420067ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc42014a318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc42014a318, 0xc4213ff000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc42014a300, 0xc4213ff000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc42014a300, 0xc4213ff000, 0x1000, 0x1000, 0xc42053cdd8, 0x1, 0xc42053ce08) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42021a510, 0xc4213ff000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc421110060, 0xc420a80030, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc421110060, 0xc420a80030, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc421110060, 0xc420a80030, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421124000, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421124000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 202 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421124000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 203 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421124000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 204 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4211140c0, 0xc420675000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc42113e4e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).ReadByte(0xc42113e4e0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421109500, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42110e0c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42113e540, 0x16e4b60, 0xc42110e120) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42113e540, 0x16e4b60, 0xc42110e120, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421118440, 0x16e4b60, 0xc42110e120, 0xc42110e120, 0xc42110c0c8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421118440, 0xc42110e120, 0x2d, 0xc420afe120) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequestHeader(0xc42110c0a0, 0x1d72340, 0xc421118440, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequest(0xc42110c0a0, 0x1d72340, 0xc421118440, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc42110c0a0, 0x1d72340, 0xc421118440, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc4211140c0, 0xc42110c0a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 167 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42039e900, 0xc4213ea000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc420d66180) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).ReadByte(0xc420d66180, 0x101000000000016, 0x0, 0x2d) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42085c960, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4201bfde0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d661e0, 0x16e4b60, 0xc4201bfec0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d661e0, 0x16e4b60, 0xc4201bfec0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420887d00, 0x16e4b60, 0xc4201bfec0, 0xc4201bfec0, 0xc420e822a8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420887d00, 0xc4201bfec0, 0x2d, 0xc4207d1520) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequestHeader(0xc420e82280, 0x1d72340, 0xc420887d00, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420438870, 0x8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequest(0xc420e82280, 0x1d72340, 0xc420887d00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc420e82280, 0x1d72340, 0xc420887d00, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc42039e900, 0xc420e82280) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 209 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42039d1e0, 0xc422d31d10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42039d1e0, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42000f408, 0xc4211fd2c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42000f408, 0xc4211fd2c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 171 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4213b4000, 0xc4213d5700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4221a61e0, 0x1d718c0, 0xc4213b4000, 0x0, 0xc4213b4000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202ae780, 0xc4213d5950, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc422867ec0, 0xc420f40700, 0xc4221a6090, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.call(0xc420452de0, 0xc42000eb38, 0x13, 0x1bda9a0, 0x4, 0xc4213d5d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.Call(0xc420452de0, 0xc42000eb38, 0x13, 0xc4213d5d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*service).call(0xc42256a400, 0xc422572050, 0xc422212e48, 0x0, 0xc421e30880, 0xc4225820a0, 0x19758e0, 0xc420f40700, 0x16, 0x169e020, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc422572050, 0x1d72340, 0xc42256a480, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc421114f00, 0xc422572050) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 231 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4218fa300, 0xc42281c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc4211aba40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).ReadByte(0xc4211aba40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42235ad50, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42229c660) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4211abaa0, 0x16e4b60, 0xc42229c6c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4211abaa0, 0x16e4b60, 0xc42229c6c0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422c32280, 0x16e4b60, 0xc42229c6c0, 0xc42229c6c0, 0xc4211fd388) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422c32280, 0xc42229c6c0, 0x2d, 0xc421c505c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequestHeader(0xc4211fd360, 0x1d72340, 0xc422c32280, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420438870, 0x8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequest(0xc4211fd360, 0x1d72340, 0xc422c32280, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc4211fd360, 0x1d72340, 0xc422c32280, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc4218fa300, 0xc4211fd360) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 173 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42039c0b0, 0xc420a4ed10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42039c0b0, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42000e798, 0xc421de60a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42000e798, 0xc421de60a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 174 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160baf0, 0x72, 0xc4211bdce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc42130e098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc42130e098, 0xc4217d3000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc42130e080, 0xc4217d3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc42130e080, 0xc4217d3000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42000e798, 0xc4217d3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc4223ee0c0, 0xc422da0000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc4223ee0c0, 0xc422da0000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc4223ee0c0, 0xc422da0000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42039c0b0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42039c0b0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 175 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42039c0b0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 176 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42039c0b0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 177 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420e7f260, 0xc420fc3700, 0x20, 0x20, 0x169d4e0, 0xc4216367a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4226fa990, 0x1d718c0, 0xc420e7f260, 0x0, 0xc420e7f200) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202ae780, 0xc420fc3950, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc422424060, 0xc421fb7180, 0xc4226fa840, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.call(0xc420452de0, 0xc42000ef78, 0x13, 0x1bda9a0, 0x4, 0xc420fc3d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.Call(0xc420452de0, 0xc42000ef78, 0x13, 0xc420fc3d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*service).call(0xc422d94440, 0xc421de6140, 0xc421bee3c8, 0x0, 0xc422205880, 0xc4220aa0e0, 0x19758e0, 0xc421fb7180, 0x16, 0x169e020, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc421de6140, 0x1d72340, 0xc422d944c0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc42039e540, 0xc421de6140) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 189 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160b470, 0x72, 0xc4211b8ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc420aea018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc420aea018, 0xc422c3b000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc420aea000, 0xc422c3b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc420aea000, 0xc422c3b000, 0x1000, 0x1000, 0xc4200505d8, 0x1, 0xc420050608) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42020e018, 0xc422c3b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc422424a20, 0xc4227bd6e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc422424a20, 0xc4227bd6e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc422424a20, 0xc4227bd6e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200cc2c0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200cc2c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 190 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200cc2c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 191 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200cc2c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 192 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420489c80, 0xc422da2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc422867980) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).ReadByte(0xc422867980, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422d4b680, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422d3e080) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4228679e0, 0x16e4b60, 0xc422d3e0e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4228679e0, 0x16e4b60, 0xc422d3e0e0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422d70540, 0x16e4b60, 0xc422d3e0e0, 0xc422d3e0e0, 0xc421de6398) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422d70540, 0xc422d3e0e0, 0x2d, 0xc42227e8c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequestHeader(0xc421de6370, 0x1d72340, 0xc422d70540, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420438870, 0x8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequest(0xc421de6370, 0x1d72340, 0xc422d70540, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc421de6370, 0x1d72340, 0xc422d70540, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc420489c80, 0xc421de6370) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 242 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420477d90, 0xc420a49d10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420477d90, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42020e800, 0xc4211bf090) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42020e800, 0xc4211bf090) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 243 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160b3a0, 0x72, 0xc422d48ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc42096d818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc42096d818, 0xc422c8c000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc42096d800, 0xc422c8c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc42096d800, 0xc422c8c000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42020e800, 0xc422c8c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc42198f980, 0xc4222bf1e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc42198f980, 0xc4222bf1e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc42198f980, 0xc4222bf1e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420477d90, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420477d90) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 244 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420477d90) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 245 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420477d90) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 246 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4213b5920, 0xc421427700, 0x20, 0x20, 0x169d4e0, 0xc4216367a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4221a7fb0, 0x1d718c0, 0xc4213b5920, 0x0, 0xc4213b5900) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202ae780, 0xc421427950, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421111c80, 0xc420f41280, 0xc4221a7ec0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.call(0xc420452de0, 0xc42020eb40, 0x13, 0x1bda9a0, 0x4, 0xc421427d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.Call(0xc420452de0, 0xc42020eb40, 0x13, 0xc421427d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*service).call(0xc422d71fc0, 0xc4211bf130, 0xc4222135b8, 0x0, 0xc420df5480, 0xc422d3f740, 0x19758e0, 0xc420f41280, 0x16, 0x169e020, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc4211bf130, 0x1d72340, 0xc422d40040, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc421114000, 0xc4211bf130) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 247 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160b2d0, 0x72, 0xc4211b9ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc421fb6118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc421fb6118, 0xc422c51000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc421fb6100, 0xc422c51000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc421fb6100, 0xc422c51000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc4204aa3f0, 0xc422c51000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc421014660, 0xc4227bd710, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc421014660, 0xc4227bd710, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc421014660, 0xc4227bd710, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420477e40, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420477e40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 248 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420477e40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 249 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420477e40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 250 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42265a240, 0xc422832000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc4216e6f60) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).ReadByte(0xc4216e6f60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422bd4030, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422d3f8e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4216e6fc0, 0x16e4b60, 0xc422d3f940) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4216e6fc0, 0x16e4b60, 0xc422d3f940, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422d40800, 0x16e4b60, 0xc422d3f940, 0xc422d3f940, 0xc4211bf298) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422d40800, 0xc422d3f940, 0x2d, 0xc42245b8a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequestHeader(0xc4211bf270, 0x1d72340, 0xc422d40800, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420438870, 0x8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequest(0xc4211bf270, 0x1d72340, 0xc422d40800, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc4211bf270, 0x1d72340, 0xc422d40800, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc42265a240, 0xc4211bf270) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 251 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420477ef0, 0xc4213c2d10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420477ef0, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42020eed0, 0xc4211bf360) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42020eed0, 0xc4211bf360) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 252 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160b200, 0x72, 0xc422d45ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc422701718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc422701718, 0xc42299e000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc422701700, 0xc42299e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc422701700, 0xc42299e000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42020eed0, 0xc42299e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc4216e7380, 0xc42287c150, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc4216e7380, 0xc42287c150, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc4216e7380, 0xc42287c150, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420477ef0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420477ef0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 253 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420477ef0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 254 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420477ef0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 255 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4211abce0, 0xc4228f3700, 0x20, 0x20, 0x169d4e0, 0xc4216367a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42235b170, 0x1d718c0, 0xc4211abce0, 0x0, 0xc4211abc00) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202ae780, 0xc4228f3950, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4216c7680, 0xc420f41f00, 0xc42235b080, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.call(0xc420452de0, 0xc42020f1d8, 0x13, 0x1bda9a0, 0x4, 0xc4228f3d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.Call(0xc420452de0, 0xc42020f1d8, 0x13, 0xc4228f3d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*service).call(0xc422d41000, 0xc4211bf400, 0xc421bee4e8, 0x0, 0xc421e31200, 0xc422d3fb60, 0x19758e0, 0xc420f41f00, 0x16, 0x169e020, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc4211bf400, 0x1d72340, 0xc422d41040, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc4218fa000, 0xc4211bf400) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 258 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff5c0f2488, 0x72, 0xc422d47ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc421e31918, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc421e31918, 0xc4217be000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc421e31900, 0xc4217be000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc421e31900, 0xc4217be000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42000f408, 0xc4217be000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc4214c6c60, 0xc421aa66a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc4214c6c60, 0xc421aa66a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc4214c6c60, 0xc421aa66a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42039d1e0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42039d1e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 259 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42039d1e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 260 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42039d1e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 261 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4211abda0, 0xc422d0d700, 0x20, 0x20, 0x169d4e0, 0xc4216367a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42235b500, 0x1d718c0, 0xc4211abda0, 0x0, 0xc4211abd00) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202ae780, 0xc422d0d950, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420087ce0, 0xc420f41f80, 0xc42235b380, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.call(0xc420452de0, 0xc42000f718, 0x13, 0x1bda9a0, 0x4, 0xc422d0dd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.Call(0xc420452de0, 0xc42000f718, 0x13, 0xc422d0dd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*service).call(0xc422c5a540, 0xc4211fd360, 0xc421bee8c8, 0x0, 0xc422da8700, 0xc422d0b080, 0x19758e0, 0xc420f41f80, 0x16, 0x169e020, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc4211fd360, 0x1d72340, 0xc422c5a580, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc4218fa180, 0xc4211fd360) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 256 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff5c0f23b8, 0x72, 0xc422cb5ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc422da8d18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc422da8d18, 0xc421cf5000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc422da8d00, 0xc421cf5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc422da8d00, 0xc421cf5000, 0x1000, 0x1000, 0xfc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42000f768, 0xc421cf5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc42122e3c0, 0xc422213f70, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc42122e3c0, 0xc422213f70, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc42122e3c0, 0xc422213f70, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421452000, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421452000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 257 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421452000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 274 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421452000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 275 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420453ec0, 0xc422aa5700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224323f0, 0x1d718c0, 0xc420453ec0, 0x0, 0xc420453e00) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202ae780, 0xc422aa5950, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420dfc600, 0xc422da8d80, 0xc4224322a0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.call(0xc420452de0, 0xc4204aadf0, 0x13, 0x1bda9a0, 0x4, 0xc422aa5d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.Call(0xc420452de0, 0xc4204aadf0, 0x13, 0xc422aa5d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*service).call(0xc422d1f780, 0xc420817450, 0xc421aa6d98, 0x0, 0xc421361480, 0xc42171e160, 0x19758e0, 0xc422da8d80, 0x16, 0x169e020, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc420817450, 0x1d72340, 0xc422d1f7c0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc421115440, 0xc420817450) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 232 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4218fa3c0, 0xc422940000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc4211abbc0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).ReadByte(0xc4211abbc0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42235ad80, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42229c7a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4211abc20, 0x16e4b60, 0xc42229c820) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4211abc20, 0x16e4b60, 0xc42229c820, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422c32300, 0x16e4b60, 0xc42229c820, 0xc42229c820, 0xc421de6168) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422c32300, 0xc42229c820, 0x2d, 0xc421c50860) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequestHeader(0xc421de6140, 0x1d72340, 0xc422c32300, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420438870, 0x8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequest(0xc421de6140, 0x1d72340, 0xc422c32300, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc421de6140, 0x1d72340, 0xc422c32300, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc4218fa3c0, 0xc421de6140) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 263 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42265a3c0, 0xc420d43000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc420453e00) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).ReadByte(0xc420453e00, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422432210, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422d0b460) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420453e60, 0x16e4b60, 0xc422d0b4c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420453e60, 0x16e4b60, 0xc422d0b4c0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422c5a980, 0x16e4b60, 0xc422d0b4c0, 0xc422d0b4c0, 0xc4211bf428) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422c5a980, 0xc422d0b4c0, 0x2d, 0xc421c50b20) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequestHeader(0xc4211bf400, 0x1d72340, 0xc422c5a980, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420438870, 0x8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequest(0xc4211bf400, 0x1d72340, 0xc422c5a980, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc4211bf400, 0x1d72340, 0xc422c5a980, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc42265a3c0, 0xc4211bf400) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 265 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42048e790, 0xc4211b6d10, 0x30, 0xf9866a) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42048e790, 0x1c61988, 0xc4202ae780, 0x1d71840, 0xc420d86f40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42000f970, 0xc4200b7400) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d320, 0xc42000f970, 0xc4200b7400) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 266 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff6160b7b0, 0x72, 0xc422cb2ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc422da9098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc422da9098, 0xc4228ed000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc422da9080, 0xc4228ed000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc422da9080, 0xc4228ed000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc42000f970, 0xc4228ed000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc42131d860, 0xc421aa7fb0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc42131d860, 0xc421aa7fb0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc42131d860, 0xc421aa7fb0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42048e790, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42048e790) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 267 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42048e790) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 268 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42048e790) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 269 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420e3b9e0, 0xc422d0f700, 0x20, 0x20, 0x169d4e0, 0xc4216367a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42153e570, 0x1d718c0, 0xc420e3b9e0, 0x0, 0xc420e3b900) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202ae780, 0xc422d0f950, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420e3b080, 0xc422289d80, 0xc42153e2d0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.call(0xc420452de0, 0xc42000fc78, 0x13, 0x1bda9a0, 0x4, 0xc422d43d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: reflect.Value.Call(0xc420452de0, 0xc42000fc78, 0x13, 0xc422d43d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*service).call(0xc422c021c0, 0xc4200b74a0, 0xc4218abbf8, 0x0, 0xc422289980, 0xc421d94340, 0x19758e0, 0xc422289d80, 0x16, 0x169e020, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc4200b74a0, 0x1d72340, 0xc422c02200, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc4218fa480, 0xc4200b74a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 279 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422321130) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 223 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200c2dc0, 0xc42265a600) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:123 +0x1fd Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c2dc0, 0xc42229d0c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 224 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200c2dc0, 0xc42265a6c0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200c2dc0, 0xc42265a6c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c2dc0, 0xc42229d0e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 294 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200c2dc0, 0xc42265a6c0, 0xc422d8d0e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c2dc0, 0xc422125300) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 306 [IO wait]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.runtime_pollWait(0x7eff5c0f22e8, 0x72, 0xc422cb1ce8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).wait(0xc421ca4e18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*pollDesc).waitRead(0xc421ca4e18, 0xc420ff5000, 0x1000, 0x1000) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: internal/poll.(*FD).Read(0xc421ca4e00, 0xc420ff5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*netFD).Read(0xc421ca4e00, 0xc420ff5000, 0x1000, 0x1000, 0xc422cb1dd8, 0x1, 0xc422cb1e08) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net.(*conn).Read(0xc4204ab050, 0xc420ff5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).Read(0xc421079620, 0xc42037f630, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadAtLeast(0x1d59aa0, 0xc421079620, 0xc42037f630, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: io.ReadFull(0x1d59aa0, 0xc421079620, 0xc42037f630, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421453b80, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421453b80) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 307 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421453b80) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 308 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421453b80) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 316 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42265aa80, 0xc420dd7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).fill(0xc421b17b60) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: bufio.(*Reader).ReadByte(0xc421b17b60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42224cdb0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421ee4980) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421b17bc0, 0x16e4b60, 0xc421ee4a00) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421b17bc0, 0x16e4b60, 0xc421ee4a00, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4229befc0, 0x16e4b60, 0xc421ee4a00, 0xc421ee4a00, 0xc420383108) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4229befc0, 0xc421ee4a00, 0x2d, 0xc421c0f620) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequestHeader(0xc4203830e0, 0x1d72340, 0xc4229befc0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420438870, 0x8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).readRequest(0xc4203830e0, 0x1d72340, 0xc4229befc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: net/rpc.(*Server).ServeRequest(0xc4203830e0, 0x1d72340, 0xc4229befc0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202ae780, 0x1d71840, 0xc420d86f40, 0x1d7d0e0, 0xc42265aa80, 0xc4203830e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 317 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4229bf340) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 303 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4207d79a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 304 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4207d79a0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 305 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc420438be0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4202ae780) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 322 [semacquire]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: sync.runtime_SemacquireMutex(0xc42033b0cc, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: sync.(*Mutex).Lock(0xc42033b0c8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: sync.(*RWMutex).Lock(0xc42033b0c8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc42033b040, 0x1d71840, 0xc42271a5c0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 323 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420486510) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 324 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420486510) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 325 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc422233800) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 326 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42271a600, 0xc422cb4ce8, 0x20, 0x20, 0x18, 0xc421d95d40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42220c030, 0x1d71840, 0xc42271a600, 0xc421d95d40, 0x184987) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4200eb4c0, 0xc4228d9ed0, 0x184987, 0x1d71840, 0xc42271a600, 0xc42096c260, 0xc4222337d0, 0xc420f6b320, 0x24, 0x2bba400) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc42096c200, 0x1d71840, 0xc42271a600, 0x184987, 0xc422984040, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc42096c200, 0x1d71840, 0xc42271a600) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 327 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42271a680, 0xc422aab770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42224d2c0, 0x1d71840, 0xc42271a680, 0x0, 0x1849b5) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4200eb4c0, 0xc422aab960, 0x1849b5, 0x1d71840, 0xc42271a680, 0x552bde, 0xc422aab950, 0xc422aab970, 0xf33ba5, 0xc420429788) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc420429730, 0x1d71840, 0xc42271a680, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc420429730) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 328 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42271a640, 0xc4228dba78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42220c240, 0x1d71840, 0xc42271a640, 0xc42220c360, 0x1833a4) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4200eb4c0, 0xc4228dbc68, 0x1833a4, 0x1d71840, 0xc42271a640, 0x552bde, 0xc420438960, 0x2, 0xc4202223c0, 0x40) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc42271a740, 0x1833a4, 0x3b, 0xc422766de8, 0x1, 0x1) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc42271a740) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 239 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422273f80, 0xc422d9d620) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 240 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422273f80, 0xc422d9d680) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 329 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc42271a780) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 330 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420f5d770, 0x1d71840, 0xc42271a640) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 283 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc420e6c6e0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: goroutine 241 [select]: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422984100, 0xc42266cd28, 0x20, 0x20, 0xc420056c00, 0x1bd9a98) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422222300, 0x1d71840, 0xc422984100, 0xc421ee4da0, 0x1849b5) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4200eb4c0, 0xc422d11f20, 0x1849b5, 0x1d71840, 0xc422984100, 0x0, 0xc422d8dce0, 0xf2c8ca, 0x1, 0x0) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc420e6c6e0, 0x1849b5, 0xc422d8dd40, 0xf2c920, 0x1, 0x0, 0x0, 0xc4222666b8) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc420e6c6e0, 0x1849b5, 0xc42188be00) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25486]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:26:56 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:26:56 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:26:56 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: ==> Starting Nomad agent... Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: ==> Nomad agent configuration: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: Client: false Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: Log Level: INFO Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: Region: amrv01 (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: Server: true Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: Version: 0.8.4 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.720945 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: Attempting re-join to previously known node: consulnomad-02.aaf01: 172.26.1.191:4648 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.721109 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.792871 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [WARN] memberlist: Refuting an alive message Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902036 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902085 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902096 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902109 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902121 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902132 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902213 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902234 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 2018/08/23 15:26:56.902247 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902489 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902502 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902514 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902525 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.902537 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56 [INFO] serf: Re-joined to previously known node: consulnomad-02.aaf01: 172.26.1.191:4648 Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.903408 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.903429 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.903448 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:26:56 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:56.903465 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 20 Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [INFO] raft: Election won. Tally: 2 Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58.446441 [INFO] nomad: cluster leadership acquired Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:58 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:58 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:59 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:59 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:59 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:59 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:59 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: 2018/08/23 15:26:59 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 1.171324695s Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: fatal error: concurrent map read and map write Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 125 [running]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc420b779c0 sp=0xc420b779a0 pc=0x42d161 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: runtime.mapaccess2(0x1844be0, 0xc42051ca80, 0xc420b77a88, 0x412539, 0xc420205c30) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc420b77a08 sp=0xc420b779c0 pc=0x409c75 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc42002ea80, 0x1d6dd40, 0xc420205c30, 0x9390f22, 0xed3108a19, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:94 +0xe2 fp=0xc420b77af8 sp=0xc420b77a08 pc=0xf20ca2 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4204f74a0, 0xc420a8bb80, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc420b77b60 sp=0xc420b77af8 pc=0xf5388e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4204f74a0, 0xc420a8bb80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc420b77b98 sp=0xc420b77b60 pc=0xf534b3 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc42058a000, 0xc4204e3501, 0xc4204e3520) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc420b77c88 sp=0xc420b77b98 pc=0xf77602 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc42058a000, 0xc420d3ad80, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc420b77d38 sp=0xc420b77c88 pc=0xf76fff Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc42058a000, 0xc420d3ad80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc420b77fa0 sp=0xc420b77d38 pc=0xf76b90 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420ba40e0, 0xc42058a000, 0xc420d3ad80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc420b77fc8 sp=0xc420b77fa0 pc=0xfb8e9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: runtime.goexit() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420b77fd0 sp=0xc420b77fc8 pc=0x45c711 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 1 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420311ce0, 0xc4200eef00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420311ce0, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc42023edc0, 0xc42023edc0, 0xc420479f80, 0x37) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4201bff68) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: main.main() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 5 [syscall]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: os/signal.signal_recv(0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: os/signal.loop() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by os/signal.init.0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 41 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc420082140, 0x1d71840, 0xc4203fdd00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 240 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc42223cf90) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 194 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42145e000, 0xc421409000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).fill(0xc420018cc0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).ReadByte(0xc420018cc0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420ec5530, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420738080) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420018d20, 0x16e4b60, 0xc4207380e0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420018d20, 0x16e4b60, 0xc4207380e0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420cb9e00, 0x16e4b60, 0xc4207380e0, 0xc4207380e0, 0xc420d360c8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420cb9e00, 0xc4207380e0, 0x2d, 0xc422f45860) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).readRequestHeader(0xc420d360a0, 0x1d72340, 0xc420cb9e00, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200a9c70, 0x8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).readRequest(0xc420d360a0, 0x1d72340, 0xc420cb9e00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).ServeRequest(0xc420d360a0, 0x1d72340, 0xc420cb9e00, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d0e0, 0xc42145e000, 0xc420d360a0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 239 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420258900) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 28 [select, locked to thread]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: runtime.selectgo(0xc420050750, 0xc4202ee960) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: runtime.ensureSigM.func1() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: runtime.goexit() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 29 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42020dfc0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 30 [sleep]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: time.Sleep(0x3b9aca00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc42023d760) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 31 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc4200f06c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 32 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc420019920) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 15 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420444790) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 66 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc420291e10) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc42051ced0, 0xc420024980) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 119 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42145ecc0, 0xc420a66000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).fill(0xc420d51260) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).ReadByte(0xc420d51260, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d65680, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4214a6dc0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d512c0, 0x16e4b60, 0xc4214a6e20) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d512c0, 0x16e4b60, 0xc4214a6e20, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420c2ef00, 0x16e4b60, 0xc4214a6e20, 0xc4214a6e20, 0xc4213dc1b8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420c2ef00, 0xc4214a6e20, 0x2d, 0xc420bc2fa0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).readRequestHeader(0xc4213dc190, 0x1d72340, 0xc420c2ef00, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9c70, 0x8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).readRequest(0xc4213dc190, 0x1d72340, 0xc420c2ef00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).ServeRequest(0xc4213dc190, 0x1d72340, 0xc420c2ef00, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d0e0, 0xc42145ecc0, 0xc4213dc190) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 182 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420444790, 0xc4205f6d10, 0x30, 0xf9866a) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420444790, 0x1c61988, 0xc42058a000, 0x1d71840, 0xc420594a80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc4205a20b0, 0xc4213dc050) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc4205a20b0, 0xc4213dc050) Aug 23 15:26:59 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 39 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a22f00, 0x72, 0xc4200609a8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc42020eb18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc42020eb18, 0xc4205cc000, 0x1000, 0x1000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Read(0xc42020eb00, 0xc4205cc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).Read(0xc42020eb00, 0xc4205cc000, 0x1000, 0x1000, 0x458600, 0xc4201eb500, 0x4) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*conn).Read(0xc4205a2008, 0xc4205cc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/http.(*persistConn).Read(0xc4203946c0, 0xc4205cc000, 0x1000, 0x1000, 0xc420060b98, 0x405675, 0xc4205d80c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).fill(0xc4205742a0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).Peek(0xc4205742a0, 0x1, 0x0, 0x0, 0x1, 0xc420d4ac60, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/http.(*persistConn).readLoop(0xc4203946c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by net/http.(*Transport).dialConn Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 236 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4200a9f40, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc42058a000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:26:59 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 120 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42041e210, 0xc420ef2d10, 0x30, 0xf9866a) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42041e210, 0x1c61988, 0xc42058a000, 0x1d71840, 0xc420594a80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc4205a3638, 0xc4213dcdc0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc4205a3638, 0xc4213dcdc0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 238 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420258900) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 14 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a22af0, 0x72, 0xc4205f7ce8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc420220498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc420220498, 0xc420022000, 0x1000, 0x1000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Read(0xc420220480, 0xc420022000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:59 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).Read(0xc420220480, 0xc420022000, 0x1000, 0x1000, 0xc420055dd8, 0x1, 0xc420055e08) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*conn).Read(0xc4205a20b0, 0xc420022000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).Read(0xc420384000, 0xc420b4e000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadAtLeast(0x1d59aa0, 0xc420384000, 0xc420b4e000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadFull(0x1d59aa0, 0xc420384000, 0xc420b4e000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420444790, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420444790) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 195 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42041e160, 0xc421475d10, 0x30, 0xf9866a) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42041e160, 0x1c61988, 0xc42058a000, 0x1d71840, 0xc420594a80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc420216490, 0xc420d36820) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc420216490, 0xc420d36820) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 161 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42041e2c0, 0xc421333d10, 0x30, 0xf9866a) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42041e2c0, 0x1c61988, 0xc42058a000, 0x1d71840, 0xc420594a80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc420204560, 0xc421092f00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc420204560, 0xc421092f00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 40 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/http.(*persistConn).writeLoop(0xc4203946c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by net/http.(*Transport).dialConn Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 16 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420444790) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 67 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420396000, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc42023c160) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 112 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200b4000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200b4000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200b4000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420068e10) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 113 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200b4000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420068e20) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 130 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200b4000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420068ed0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 131 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420f00420) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 132 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc42058e680) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 133 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a22d60, 0x72, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc420220218, 0x72, 0xc4201fe200, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc420220218, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Accept(0xc420220200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).accept(0xc420220200, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*TCPListener).accept(0xc420204070, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*TCPListener).AcceptTCP(0xc420204070, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420f103c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 134 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a22c90, 0x72, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc420220318, 0x72, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc420220318, 0xc421f1a000, 0x10000, 0x10000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).ReadFrom(0xc420220300, 0xc421f1a000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).readFrom(0xc420220300, 0xc421f1a000, 0x10000, 0x10000, 0x13a, 0x13a, 0x1d62b40, 0x412539, 0xc421334e40) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*UDPConn).readFrom(0xc420204080, 0xc421f1a000, 0x10000, 0x10000, 0x101, 0xc421334e18, 0xc421334e90, 0x443cf7) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*UDPConn).ReadFrom(0xc420204080, 0xc421f1a000, 0x10000, 0x10000, 0x10000, 0x10000, 0x594b, 0xa1b5e90d, 0x2b96de0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420f103c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 135 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420f103c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 136 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420f103c0, 0x12a05f200, 0xc420dfca80, 0xc420cc5740, 0xc42067e370) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 137 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420f103c0, 0xc420cc5740) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 138 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420f103c0, 0x1dcd6500, 0xc420dfcae0, 0xc420cc5740, 0xc42067e380) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 139 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4204d6600) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 140 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4204d6600) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 141 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204d6600, 0x1bdd09a, 0x6, 0xc420f00570) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 142 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204d6600, 0x1bdba6c, 0x5, 0xc420f005a0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 143 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204d6600, 0x1bdbc6a, 0x5, 0xc420f005d0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 171 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200be2c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 145 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420dfcb40, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420dfcb40, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420dfcb40) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 146 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420dfcba0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420dfcba0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420dfcba0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 147 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420dfcc00, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420dfcc00, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420dfcc00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 148 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420dfcc60, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420dfcc60, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420dfcc60) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 149 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420a9a5d0, 0xc42058a000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 150 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc42058a000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 151 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc42058a000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 152 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a22e30, 0x72, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc42020ef98, 0x72, 0xc421330200, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc42020ef98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Accept(0xc42020ef80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).accept(0xc42020ef80, 0x1, 0x1, 0xc421332f20) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*TCPListener).accept(0xc4205a23b0, 0x406b4a, 0xc420d607e0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*TCPListener).Accept(0xc4205a23b0, 0xc420d607e0, 0x0, 0x3f800000, 0xc420594a80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc42058a000, 0x1d71840, 0xc420594a80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 153 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4204f74a0, 0x3b9aca00, 0xc4202eede0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 154 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4200a9f40, 0x3b9aca00, 0xc4202eede0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 155 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420258900, 0x3b9aca00, 0xc4202eede0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 156 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc420291e10, 0x3b9aca00, 0xc4202eede0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 157 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc42058a000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 158 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a22bc0, 0x72, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc420221298, 0x72, 0xc4203fcf00, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc420221298, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Accept(0xc420221280, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).accept(0xc420221280, 0x402037, 0xc4207a4080, 0x1805ec0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*TCPListener).accept(0xc420204538, 0x458e40, 0xc4205f5e90, 0xc4205f5e98) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*TCPListener).Accept(0xc420204538, 0x1c65af8, 0xc4207a4000, 0x1d71900, 0xc42133e0f0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/http.(*Server).Serve(0xc420d22000, 0x1d6fc00, 0xc420204538, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/http.Serve(0x1d6fc00, 0xc420204538, 0x1d5df00, 0xc42133e030, 0x706d6f4364657061, 0x73616c4364657475) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420594b40, 0x1d6fc00, 0xc420204538, 0xc42067ea90, 0xc420a9a690) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 159 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420221380, 0xc42067eaa0, 0xc420cc5da0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 235 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4208566e0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 178 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a22a20, 0x72, 0xc4205b0ce8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc420221418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc420221418, 0xc4213fe000, 0x1000, 0x1000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Read(0xc420221400, 0xc4213fe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).Read(0xc420221400, 0xc4213fe000, 0x1000, 0x1000, 0xc4205a95d8, 0x1, 0xc4205a9608) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*conn).Read(0xc420204560, 0xc4213fe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).Read(0xc420dfcf60, 0xc4209100c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadAtLeast(0x1d59aa0, 0xc420dfcf60, 0xc4209100c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadFull(0x1d59aa0, 0xc420dfcf60, 0xc4209100c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42041e2c0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42041e2c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 179 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42041e2c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 180 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42041e2c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 181 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4202f83c0, 0xc4213ff000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).fill(0xc4214052c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).ReadByte(0xc4214052c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a9bd10, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207cd2c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421405320, 0x16e4b60, 0xc4207cd320) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421405320, 0x16e4b60, 0xc4207cd320, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420595080, 0x16e4b60, 0xc4207cd320, 0xc4207cd320, 0xc421093018) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420595080, 0xc4207cd320, 0x2d, 0xc422f458e0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).readRequestHeader(0xc421092ff0, 0x1d72340, 0xc420595080, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a9c70, 0x8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).readRequest(0xc421092ff0, 0x1d72340, 0xc420595080, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).ServeRequest(0xc421092ff0, 0x1d72340, 0xc420595080, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d0e0, 0xc4202f83c0, 0xc421092ff0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 183 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42041e0b0, 0xc42146fd10, 0x30, 0xf9866a) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42041e0b0, 0x1c61988, 0xc42058a000, 0x1d71840, 0xc420594a80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc4205a20e8, 0xc4213dc0f0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc4205a20e8, 0xc4213dc0f0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 184 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a22950, 0x72, 0xc421339ce8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc420220618, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc420220618, 0xc420ab0000, 0x1000, 0x1000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Read(0xc420220600, 0xc420ab0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).Read(0xc420220600, 0xc420ab0000, 0x1000, 0x1000, 0xa8, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*conn).Read(0xc4205a20e8, 0xc420ab0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).Read(0xc4200863c0, 0xc420c4e270, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadAtLeast(0x1d59aa0, 0xc4200863c0, 0xc420c4e270, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadFull(0x1d59aa0, 0xc4200863c0, 0xc420c4e270, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42041e0b0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42041e0b0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 185 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42041e0b0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 186 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42041e0b0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 187 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42145e300, 0xc420ab1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).fill(0xc420575740) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).ReadByte(0xc420575740, 0x100000000000001, 0x1000000000000, 0xd40) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4208eb5c0, 0x413e07) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4214a6080) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4205757a0, 0x16e4b60, 0xc4214a60e0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4205757a0, 0x16e4b60, 0xc4214a60e0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420cff100, 0x16e4b60, 0xc4214a60e0, 0xc4214a60e0, 0xc4213dc1b8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420cff100, 0xc4214a60e0, 0x2d, 0xc4206e2a40) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).readRequestHeader(0xc4213dc190, 0x1d72340, 0xc420cff100, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9c70, 0x8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).readRequest(0xc4213dc190, 0x1d72340, 0xc420cff100, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).ServeRequest(0xc4213dc190, 0x1d72340, 0xc420cff100, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d0e0, 0xc42145e300, 0xc4213dc190) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 188 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a22880, 0x72, 0xc421479ce8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc421407c98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc421407c98, 0xc420933000, 0x1000, 0x1000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Read(0xc421407c80, 0xc420933000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).Read(0xc421407c80, 0xc420933000, 0x1000, 0x1000, 0xc4204ad5d8, 0x1, 0xc4204ad608) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*conn).Read(0xc420216490, 0xc420933000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).Read(0xc420575c80, 0xc420b4e010, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadAtLeast(0x1d59aa0, 0xc420575c80, 0xc420b4e010, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadFull(0x1d59aa0, 0xc420575c80, 0xc420b4e010, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42041e160, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42041e160) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 189 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42041e160) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 190 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42041e160) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 191 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4202f8000, 0xc420bd8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).fill(0xc420d4d080) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).ReadByte(0xc420d4d080, 0x100000000000016, 0x0, 0x2d) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d6e690, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4214a6320) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d4d0e0, 0x16e4b60, 0xc4214a6380) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d4d0e0, 0x16e4b60, 0xc4214a6380, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420c89bc0, 0x16e4b60, 0xc4214a6380, 0xc4214a6380, 0xc4213dc6b8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420c89bc0, 0xc4214a6380, 0x2d, 0xc422f45960) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).readRequestHeader(0xc4213dc690, 0x1d72340, 0xc420c89bc0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a9c70, 0x8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).readRequest(0xc4213dc690, 0x1d72340, 0xc420c89bc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).ServeRequest(0xc4213dc690, 0x1d72340, 0xc420c89bc0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d0e0, 0xc4202f8000, 0xc4213dc690) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 216 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200be2c0, 0xc420b74d10, 0x30, 0xf9866a) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200be2c0, 0x1c61988, 0xc42058a000, 0x1d71840, 0xc420594a80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc4205a23e8, 0xc4210d8140) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc4205a23e8, 0xc4210d8140) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 121 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a227b0, 0x72, 0xc42146ece8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc421407f98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc421407f98, 0xc420aba000, 0x1000, 0x1000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Read(0xc421407f80, 0xc420aba000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).Read(0xc421407f80, 0xc420aba000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*conn).Read(0xc4205a3638, 0xc420aba000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).Read(0xc420d51800, 0xc420cf6ea0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadAtLeast(0x1d59aa0, 0xc420d51800, 0xc420cf6ea0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadFull(0x1d59aa0, 0xc420d51800, 0xc420cf6ea0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42041e210, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42041e210) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 122 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42041e210) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 123 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42041e210) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 124 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421389ec0, 0xc420eb9700, 0x20, 0x20, 0x169d4e0, 0xc4204e36e0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420838ea0, 0x1d718c0, 0xc421389ec0, 0x0, 0xc421389e00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42058a000, 0xc420eb9950, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421451920, 0xc420d6b080, 0xc420838c30, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: reflect.Value.call(0xc42059dbc0, 0xc4205a3940, 0x13, 0x1bda9a0, 0x4, 0xc420eb9d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: reflect.Value.Call(0xc42059dbc0, 0xc4205a3940, 0x13, 0xc420eb9d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*service).call(0xc420c2f480, 0xc4213dce60, 0xc420cdc188, 0x0, 0xc420d6a880, 0xc4214a7000, 0x19758e0, 0xc420d6b080, 0x16, 0x169e020, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).ServeRequest(0xc4213dce60, 0x1d72340, 0xc420c2f4c0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d0e0, 0xc4202f8480, 0xc4213dce60) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 193 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420444840, 0xc42147ad10, 0x30, 0xf9866a) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420444840, 0x1c61988, 0xc42058a000, 0x1d71840, 0xc420594a80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc420216498, 0xc420cbb220) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d320, 0xc420216498, 0xc420cbb220) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 210 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a226e0, 0x72, 0xc420b75ce8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc420e94e98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc420e94e98, 0xc4207d3000, 0x1000, 0x1000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Read(0xc420e94e80, 0xc4207d3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).Read(0xc420e94e80, 0xc4207d3000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*conn).Read(0xc420216498, 0xc4207d3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).Read(0xc4201f7440, 0xc4209101d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadAtLeast(0x1d59aa0, 0xc4201f7440, 0xc4209101d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadFull(0x1d59aa0, 0xc4201f7440, 0xc4209101d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420444840, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420444840) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 211 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420444840) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 212 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420444840) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 213 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420e86720, 0xc420eb5700, 0x20, 0x20, 0x169d4e0, 0xc420ee4a00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420dca900, 0x1d718c0, 0xc420e86720, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42058a000, 0xc420eb5950, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420e7d440, 0xc420938200, 0xc420ce9110, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: reflect.Value.call(0xc42059dbc0, 0xc4202167a0, 0x13, 0x1bda9a0, 0x4, 0xc420eb5d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: reflect.Value.Call(0xc42059dbc0, 0xc4202167a0, 0x13, 0xc420eb5d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*service).call(0xc420c28f80, 0xc420cbb360, 0xc4207dd828, 0x0, 0xc420cc7780, 0xc42074a900, 0x19758e0, 0xc420938200, 0x16, 0x169e020, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).ServeRequest(0xc420cbb360, 0x1d72340, 0xc420c29600, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d0e0, 0xc42145f080, 0xc420cbb360) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 166 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4200b4000, 0xc4205ea300, 0x185fc5, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:171 +0x7a2 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b4000, 0xc4205ea300) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:135 +0x258 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420739380) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 172 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200be2c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 167 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b4000, 0xc4205ea540, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b4000, 0xc4205ea540) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4207393a0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 126 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b4000, 0xc4205ea300, 0xc420d3af00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:351 +0x4fe Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4214a7900) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 129 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420b50840) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 128 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b4000, 0xc4205ea540, 0xc420c90e40) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4214a7a20) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 226 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b4000, 0xc4205ea540, 0x1d6eb80, 0xc420b50840, 0xc420c91020, 0xc420c911a0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420b50c80) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 170 [IO wait]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.runtime_pollWait(0x7f7443a22470, 0x72, 0xc420c69ce8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).wait(0xc420caa018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*pollDesc).waitRead(0xc420caa018, 0xc420f45000, 0x1000, 0x1000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: internal/poll.(*FD).Read(0xc420caa000, 0xc420f45000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*netFD).Read(0xc420caa000, 0xc420f45000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net.(*conn).Read(0xc4205a23e8, 0xc420f45000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: bufio.(*Reader).Read(0xc4213ee060, 0xc420ff0190, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadAtLeast(0x1d59aa0, 0xc4213ee060, 0xc420ff0190, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: io.ReadFull(0x1d59aa0, 0xc4213ee060, 0xc420ff0190, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200be2c0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200be2c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 173 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4213888a0, 0xc420d33700, 0x20, 0x20, 0x169d4e0, 0xc420ee4a00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4213f6c90, 0x1d718c0, 0xc4213888a0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42058a000, 0xc420d33950, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420d507e0, 0xc420caa080, 0xc420cd4f30, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: reflect.Value.call(0xc42059dbc0, 0xc420205108, 0x13, 0x1bda9a0, 0x4, 0xc420d33d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: reflect.Value.Call(0xc42059dbc0, 0xc420205108, 0x13, 0xc420d33d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*service).call(0xc4209bf100, 0xc4212140a0, 0xc420b3e2e8, 0x0, 0xc420dcf000, 0xc4212f80a0, 0x19758e0, 0xc420caa080, 0x16, 0x169e020, ...) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: net/rpc.(*Server).ServeRequest(0xc4212140a0, 0x1d72340, 0xc4209bf1c0, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42058a000, 0x1d71840, 0xc420594a80, 0x1d7d0e0, 0xc4205ea0c0, 0xc4212140a0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 234 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4208566e0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 237 [semacquire]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: sync.runtime_SemacquireMutex(0xc4204f752c, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: sync.(*Mutex).Lock(0xc4204f7528) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: sync.(*RWMutex).Lock(0xc4204f7528) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4204f74a0, 0x1d71840, 0xc422837300) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 175 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4227a8030, 0xc4227ee180) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 241 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422837340, 0xc420c64ce8, 0x20, 0x20, 0x18, 0xc421e102a0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4227a8240, 0x1d71840, 0xc422837340, 0xc421e102a0, 0x184987) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204e3520, 0xc422e95ed0, 0x184987, 0x1d71840, 0xc422837340, 0xc4202205e0, 0xc42223cf30, 0xc420eda9c0, 0x24, 0x2bba400) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420220580, 0x1d71840, 0xc422837340, 0x184987, 0xc42280e540, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420220580, 0x1d71840, 0xc422837340) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 258 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422837400, 0xc420eb3770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4227a8390, 0x1d71840, 0xc422837400, 0x0, 0x1849b5) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204e3520, 0xc420eb3960, 0x1849b5, 0x1d71840, 0xc422837400, 0x552bde, 0xc420eb3950, 0xc420eb3970, 0xf33ba5, 0xc4202564b8) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc420256460, 0x1d71840, 0xc422837400, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc420256460) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 259 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422837380, 0xc422e91a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4223b82d0, 0x1d71840, 0xc422837380, 0xc4223b83f0, 0x1833a4) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204e3520, 0xc422e91c68, 0x1833a4, 0x1d71840, 0xc422837380, 0x552bde, 0xc4200a9d10, 0x2, 0xc4205542c0, 0x40) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422837440, 0x1833a4, 0x3b, 0xc420b79de8, 0x1, 0x1) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422837440) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 260 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc4228374c0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 261 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420f105a0, 0x1d71840, 0xc422837380) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 176 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4227a8030, 0xc4227ee1e0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 177 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc420856000) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: goroutine 243 [select]: Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42280e740, 0xc420c67d28, 0x20, 0x20, 0xc4204d4000, 0x1bd9a98) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4223a02d0, 0x1d71840, 0xc42280e740, 0xc420f782c0, 0x1849b5) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204e3520, 0xc42134bf20, 0x1849b5, 0x1d71840, 0xc42280e740, 0xc, 0xc, 0x0, 0x0, 0xc420055fa0) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc420856000, 0x1849b5, 0x0, 0xc, 0xc, 0xc420b4e000, 0xc420cc5bc0, 0x71ce7c) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc420856000, 0x1849b5, 0xc42167e240) Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:26:59 ip-172-24-20-236 nomad.sh[25507]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:26:59 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:26:59 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:26:59 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: ==> Starting Nomad agent... Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: ==> Nomad agent configuration: Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: Client: false Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: Log Level: INFO Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: Region: amrv01 (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: Server: true Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: Version: 0.8.4 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.223093 [INFO] nomad: starting 4 scheduling worker(s) for [system service batch _core] Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: Attempting re-join to previously known node: ip-172-25-5-80.amv01: 172.25.5.80:4648 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.225205 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.577998 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578022 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578036 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578080 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578312 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578325 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578337 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578348 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578360 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578371 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578410 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.578425 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [WARN] memberlist: Refuting an alive message Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00 [INFO] serf: Re-joined to previously known node: ip-172-25-5-80.amv01: 172.25.5.80:4648 Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.579006 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.579019 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.579030 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.579047 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.579058 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.579069 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:00 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:00.579080 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:01 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:01 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 22 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03.377123 [INFO] nomad: cluster leadership acquired Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: 2018/08/23 15:27:03 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: fatal error: concurrent map read and map write Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 270 [running]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc42212cd20 sp=0xc42212cd00 pc=0x42d161 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: runtime.mapaccess2(0x1844be0, 0xc4201f7f20, 0xc42212cde0, 0xc4204a3bb0, 0x8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc42212cd68 sp=0xc42212cd20 pc=0x409c75 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc42027faa0, 0x1d6dd40, 0xc4204a3bb0, 0x0, 0xed3109f17) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:156 +0xdc fp=0xc42212ce50 sp=0xc42212cd68 pc=0xf215ac Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4203fa0a0, 0x1d71840, 0xc4222639c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc42212cfc8 sp=0xc42212ce50 pc=0xf56bb7 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: runtime.goexit() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc42212cfd0 sp=0xc42212cfc8 pc=0x45c711 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 1 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42022ae00, 0xc4203e4000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42022ae00, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4202ac000, 0xc4202ac000, 0xc420076ee0, 0x37) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204d7f68) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: main.main() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 5 [syscall]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: os/signal.signal_recv(0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: os/signal.loop() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by os/signal.init.0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 251 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42018be00, 0xc422c46000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).fill(0xc422702f60) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).ReadByte(0xc422702f60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422227440, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4213677c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422702fc0, 0x16e4b60, 0xc421367820) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422702fc0, 0x16e4b60, 0xc421367820, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4223beb80, 0x16e4b60, 0xc421367820, 0xc421367820, 0xc422c9cd48) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4223beb80, 0xc421367820, 0x2d, 0xc4224e4920) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequestHeader(0xc422c9cd20, 0x1d72340, 0xc4223beb80, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420201590, 0x8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequest(0xc422c9cd20, 0x1d72340, 0xc4223beb80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc422c9cd20, 0x1d72340, 0xc4223beb80, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc42018be00, 0xc422c9cd20) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 137 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420299290, 0xc420514d10, 0x30, 0xf9866a) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420299290, 0x1c61988, 0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42000e008, 0xc4214ee500) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42000e008, 0xc4214ee500) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 36 [sleep]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: time.Sleep(0x3b9aca00) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420188000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 125 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4205da000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205da000, 0xc4202fb400) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 35 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42020ae40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 144 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b2537b0, 0x72, 0xc42115ece8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc42143e018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc42143e018, 0xc420c38000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc42143e000, 0xc420c38000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc42143e000, 0xc420c38000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc42020e250, 0xc420c38000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc4201f49c0, 0xc420ddf7f0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc4201f49c0, 0xc420ddf7f0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc4201f49c0, 0xc420ddf7f0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202994a0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202994a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 55 [select, locked to thread]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: runtime.selectgo(0xc420050f50, 0xc4200907e0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: runtime.ensureSigM.func1() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: runtime.goexit() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 37 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc42018a300) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 38 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4204aa4e0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 140 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420299290) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 40 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4202d4750) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0xc4200fc000, 0xc4200fc020) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc420194390, 0xc42017ec70) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 21 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4204e5040, 0x1d71840, 0xc42020aec0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 185 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420ef73e0, 0xc42283f208, 0x20, 0x20, 0xc420fb9a80, 0xc422c457a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422c45bc0, 0x1d718c0, 0xc420ef73e0, 0x0, 0xc420ef7300) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202a8000, 0xc42283f440, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Job).List(0xc4202061b8, 0xc420ef6420, 0xc422c457a0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/job_endpoint.go:925 +0x2ea Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.call(0xc4204ab620, 0xc4202066a0, 0x13, 0x1bda9a0, 0x4, 0xc421221898, 0x3, 0x3, 0x192b7e0, 0x19, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.Call(0xc4204ab620, 0xc4202066a0, 0x13, 0xc421def898, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*service).call(0xc4204e51c0, 0xc420300c30, 0xc4226a46a0, 0x0, 0xc420135a80, 0xc4205e6020, 0x1974e60, 0xc420ef6420, 0x16, 0x169dd20, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc420300c30, 0x1d71b80, 0xc42177cbc0, 0xc4201e6400, 0xa) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc4202a8000, 0x1be08be, 0x8, 0x1974e60, 0xc420ef6360, 0x169dd20, 0xc422c45770, 0xc420ef6301, 0xc422c45770) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc42013e0e0, 0x1be08be, 0x8, 0x1974e60, 0xc420ef6360, 0x169dd20, 0xc422c45770, 0x0, 0xc4201e6400) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).jobListRequest(0xc420fafe00, 0x1d6df80, 0xc420ef6300, 0xc420cca000, 0x44774f, 0xc42028b500, 0xc42030a070, 0x1d) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:33 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest(0xc420fafe00, 0x1d6df80, 0xc420ef6300, 0xc420cca000, 0xc42030a0a0, 0xc421defb58, 0x38, 0xc421defb50) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:18 +0x1a6 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest-fm(0x1d6df80, 0xc420ef6300, 0xc420cca000, 0xc420cca000, 0xc4218e01e0, 0x1d, 0xbed7c7e58389f0bb) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:141 +0x48 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc420ef6300, 0xc420cca000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.HandlerFunc.ServeHTTP(0xc420ab6b80, 0x1d6df80, 0xc420ef6300, 0xc420cca000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.(*ServeMux).ServeHTTP(0xc4213923f0, 0x1d6df80, 0xc420ef6300, 0xc420cca000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc4221d20e0, 0xc420cca000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.HandlerFunc.ServeHTTP(0xc42139e0f0, 0x1d6fec0, 0xc4221d20e0, 0xc420cca000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.serverHandler.ServeHTTP(0xc4202d4270, 0x1d6fec0, 0xc4221d20e0, 0xc420cca000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.(*conn).serve(0xc421aa4000, 0x1d71840, 0xc42177ca80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by net/http.(*Server).Serve Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 64 [select]: Aug 23 15:27:03 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42179ba20) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 139 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420299290) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 123 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4205da000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4205da000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4205da000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205da000, 0xc4202fb3e0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 60 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253f00, 0x72, 0xc4200669a8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc420393898, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc420393898, 0xc420363000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc420393880, 0xc420363000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc420393880, 0xc420363000, 0x1000, 0x1000, 0x458600, 0xc420488780, 0x4) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc4204a2590, 0xc420363000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.(*persistConn).Read(0xc42002ec60, 0xc420363000, 0x1000, 0x1000, 0xc420066b98, 0x405675, 0xc420090d20) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).fill(0xc4201f5800) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Peek(0xc4201f5800, 0x1, 0x0, 0x0, 0x1, 0xc420ec1920, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.(*persistConn).readLoop(0xc42002ec60) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by net/http.(*Transport).dialConn Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 61 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.(*persistConn).writeLoop(0xc42002ec60) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by net/http.(*Transport).dialConn Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 141 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42044e240, 0xc420738000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).fill(0xc420317980) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).ReadByte(0xc420317980, 0x101000000000016, 0x0, 0x2d) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:03 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a1ca80, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421404860) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4203179e0, 0x16e4b60, 0xc4214048c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4203179e0, 0x16e4b60, 0xc4214048c0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420c1ac40, 0x16e4b60, 0xc4214048c0, 0xc4214048c0, 0xc4214ee5c8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420c1ac40, 0xc4214048c0, 0x2d, 0xc420750200) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequestHeader(0xc4214ee5a0, 0x1d72340, 0xc420c1ac40, 0x30, 0x1, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequest(0xc4214ee5a0, 0x1d72340, 0xc420c1ac40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc4214ee5a0, 0x1d72340, 0xc420c1ac40, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc42044e240, 0xc4214ee5a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 186 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420ef7c80, 0xc4228432c8, 0x20, 0x20, 0x5, 0x1bd9a98) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421ec63f0, 0x1d718c0, 0xc420ef7c80, 0x0, 0xc420ef7c00) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202a8000, 0xc422843500, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Node).List(0xc42056a540, 0xc420f779e0, 0xc421eb0180, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:1198 +0x27c Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.call(0xc42056a840, 0xc420206d28, 0x13, 0x1bda9a0, 0x4, 0xc4213c18f8, 0x3, 0x3, 0x192ba60, 0x19, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.Call(0xc42056a840, 0xc420206d28, 0x13, 0xc42115a8f8, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*service).call(0xc4204e54c0, 0xc420300c30, 0xc422382580, 0x0, 0xc420537200, 0xc42082c0e0, 0x1975760, 0xc420f779e0, 0x16, 0x169e120, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc420300c30, 0x1d71b80, 0xc42235e840, 0xc4201b2400, 0xa) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc4202a8000, 0x1be205d, 0x9, 0x1975760, 0xc420f77920, 0x169e120, 0xc421eb00f0, 0xc420f77901, 0xc421eb00f0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc42013e0e0, 0x1be205d, 0x9, 0x1975760, 0xc420f77920, 0x169e120, 0xc421eb00f0, 0xc420485500, 0x1e) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest(0xc420fafe00, 0x1d6df80, 0xc420f778c0, 0xc4209da300, 0xc420485530, 0xc42115ab58, 0x38, 0xc42115ab50) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/node_endpoint.go:24 +0x1b3 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest-fm(0x1d6df80, 0xc420f778c0, 0xc4209da300, 0xc4209da300, 0xc421d35400, 0x1e, 0xbed7c7e58389efb2) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:145 +0x48 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc420f778c0, 0xc4209da300) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.HandlerFunc.ServeHTTP(0xc420ab6c00, 0x1d6df80, 0xc420f778c0, 0xc4209da300) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.(*ServeMux).ServeHTTP(0xc4213923f0, 0x1d6df80, 0xc420f778c0, 0xc4209da300) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc4213ea620, 0xc4209da300) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.HandlerFunc.ServeHTTP(0xc42139e0f0, 0x1d6fec0, 0xc4213ea620, 0xc4209da300) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.serverHandler.ServeHTTP(0xc4202d4270, 0x1d6fec0, 0xc4213ea620, 0xc4209da300) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.(*conn).serve(0xc421aa40a0, 0x1d71840, 0xc42235e6c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by net/http.(*Server).Serve Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 41 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420300e10, 0x1c636a0, 0xc420188160, 0x1d71840, 0xc420580280) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc420188160) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 138 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253950, 0x72, 0xc421160ce8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc420f37d18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc420f37d18, 0xc4204e1000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc420f37d00, 0xc4204e1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc420f37d00, 0xc4204e1000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc42000e008, 0xc4204e1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc421492c60, 0xc420440010, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc421492c60, 0xc420440010, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc421492c60, 0xc420440010, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420299290, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420299290) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 126 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420e0c180) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 124 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4205da000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205da000, 0xc4202fb3f0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 143 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202994a0, 0xc420515d10, 0x30, 0xf9866a) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202994a0, 0x1c61988, 0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42020e250, 0xc42108afa0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42020e250, 0xc42108afa0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 62 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42179ba20, 0xc420eded10, 0x30, 0xf9866a) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42179ba20, 0x1c61988, 0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc420206fc8, 0xc42118d450) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc420206fc8, 0xc42118d450) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 127 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4202f0410) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 128 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253d60, 0x72, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc420cdc018, 0x72, 0xc42020a000, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc420cdc018, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Accept(0xc420cdc000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).accept(0xc420cdc000, 0xc420207608, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*TCPListener).accept(0xc42000e138, 0x458e40, 0xc4204b4f90, 0xc4204b4f98) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*TCPListener).AcceptTCP(0xc42000e138, 0x1c63658, 0xc420fca5a0, 0xc420207608) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420fca5a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 129 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253c90, 0x72, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc420cdc098, 0x72, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc420cdc098, 0xc422fa4000, 0x10000, 0x10000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).ReadFrom(0xc420cdc080, 0xc422fa4000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).readFrom(0xc420cdc080, 0xc422fa4000, 0x10000, 0x10000, 0xe9, 0xe9, 0x1d62b40, 0x412539, 0xc4213a1e40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*UDPConn).readFrom(0xc42000e408, 0xc422fa4000, 0x10000, 0x10000, 0x101, 0xc4213a1e18, 0xc4213a1e90, 0x443cf7) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*UDPConn).ReadFrom(0xc42000e408, 0xc422fa4000, 0x10000, 0x10000, 0x10000, 0x10000, 0xa63e, 0xd9f1916e, 0x2b96de0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420fca5a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 146 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420fca5a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 147 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420fca5a0, 0x12a05f200, 0xc4214a3620, 0xc420f6d260, 0xc4202fbbf0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:110 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 148 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420fca5a0, 0xc420f6d260) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 149 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420fca5a0, 0x1dcd6500, 0xc4214a3680, 0xc420f6d260, 0xc4202fbc10) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 150 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc42085e480) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 151 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc42085e480) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 152 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42085e480, 0x1bdd09a, 0x6, 0xc420e0c2d0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 153 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42085e480, 0x1bdba6c, 0x5, 0xc420e0c300) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 154 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42085e480, 0x1bdbc6a, 0x5, 0xc420e0c330) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 63 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253540, 0x72, 0xc421161ce8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc422358518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc422358518, 0xc4224da000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc422358500, 0xc4224da000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc422358500, 0xc4224da000, 0x1000, 0x1000, 0xc421161dd8, 0x0, 0xc421161e08) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc420206fc8, 0xc4224da000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc421540660, 0xc4226a5400, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc421540660, 0xc4226a5400, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc421540660, 0xc4226a5400, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42179ba20, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42179ba20) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 156 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4214a36e0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214a36e0, 0x1dcd6500, 0x1bd9928, 0x2, 0x1bdab30, 0x4, 0xc421462000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214a36e0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 157 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4214a3740, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214a3740, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214a3740) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 158 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4214a37a0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214a37a0, 0x1dcd6500, 0x1bd9928, 0x2, 0x1bd9e3a, 0x3, 0xc421022300) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214a37a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 159 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4214a3800, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214a3800, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214a3800) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 160 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc421392330, 0xc4202a8000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 161 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4202a8000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 162 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4202a8000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 163 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253e30, 0x72, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc420537518, 0x72, 0xc420f74000, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc420537518, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Accept(0xc420537500, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).accept(0xc420537500, 0x1, 0x1, 0xc420ee3f20) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*TCPListener).accept(0xc420206dd8, 0x406b4a, 0xc420ffc360, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*TCPListener).Accept(0xc420206dd8, 0xc420ffc360, 0x0, 0x3f800000, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 164 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4203fa0a0, 0x3b9aca00, 0xc4202de420) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 165 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc420300be0, 0x3b9aca00, 0xc4202de420) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 166 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc42018e120, 0x3b9aca00, 0xc4202de420) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 167 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4202d4750, 0x3b9aca00, 0xc4202de420) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 168 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4202a8000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 169 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253bc0, 0x72, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc420cdc798, 0x72, 0xc420594100, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc420cdc798, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Accept(0xc420cdc780, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).accept(0xc420cdc780, 0x402037, 0xc421aa4120, 0x1805ec0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*TCPListener).accept(0xc42000e708, 0x458e40, 0xc420519e90, 0xc420519e98) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*TCPListener).Accept(0xc42000e708, 0x1c65af8, 0xc421aa40a0, 0x1d71900, 0xc42139e1b0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.(*Server).Serve(0xc4202d4270, 0x1d6fc00, 0xc42000e708, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.Serve(0x1d6fc00, 0xc42000e708, 0x1d5df00, 0xc42139e0f0, 0xc421398d80, 0x1d830c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420fafe00, 0x1d6fc00, 0xc42000e708, 0xc42052e300, 0xc4213923f0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 170 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420cdc880, 0xc42052e320, 0xc420f6d9e0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 274 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4229d7860, 0xc42293f700, 0x20, 0x20, 0x169d4e0, 0xc4205748c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4222bb4d0, 0x1d718c0, 0xc4229d7860, 0x0, 0xc4229d7800) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202a8000, 0xc42293f950, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420ab9ec0, 0xc422a91700, 0xc4222bb3b0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.call(0xc42056a780, 0xc42020f280, 0x13, 0x1bda9a0, 0x4, 0xc421dedd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.Call(0xc42056a780, 0xc42020f280, 0x13, 0xc421dedd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*service).call(0xc42235f1c0, 0xc422c9cd20, 0xc422248f58, 0x0, 0xc42187ca80, 0xc4219818c0, 0x19758e0, 0xc422a91700, 0x16, 0x169e020, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc422c9cd20, 0x1d72340, 0xc422314640, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc421475140, 0xc422c9cd20) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 145 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202994a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 178 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202994a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 179 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420d36de0, 0xc4213c5700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4210b42d0, 0x1d718c0, 0xc420d36de0, 0x0, 0xc420d36d00) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202a8000, 0xc4213c5950, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420317aa0, 0xc420fc8180, 0xc4210b4120, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.call(0xc42056a780, 0xc42020e9c8, 0x13, 0x1bda9a0, 0x4, 0xc4213c5d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.Call(0xc42056a780, 0xc42020e9c8, 0x13, 0xc4213c5d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*service).call(0xc42113bfc0, 0xc42108b040, 0xc420a669c8, 0x0, 0xc420d27a80, 0xc420da88a0, 0x19758e0, 0xc420fc8180, 0x16, 0x169e020, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc42108b040, 0x1d72340, 0xc4201fc280, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc4214740c0, 0xc42108b040) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 176 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4209f20c0, 0xc420f12000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).fill(0xc421507f80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).ReadByte(0xc421507f80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d3ad50, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4200dea80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420086180, 0x16e4b60, 0xc4200deae0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420086180, 0x16e4b60, 0xc4200deae0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421a5db80, 0x16e4b60, 0xc4200deae0, 0xc4200deae0, 0xc420759ba8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421a5db80, 0xc4200deae0, 0x2d, 0xc420c838a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequestHeader(0xc420759b80, 0x1d72340, 0xc421a5db80, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420201590, 0x8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequest(0xc420759b80, 0x1d72340, 0xc421a5db80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc420759b80, 0x1d72340, 0xc421a5db80, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc4209f20c0, 0xc420759b80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 104 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421920000, 0xc420fdbd10, 0x30, 0xf9866a) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421920000, 0x1c61988, 0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42020e010, 0xc420e0e820) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42020e010, 0xc420e0e820) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 105 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b2536e0, 0x72, 0xc421df2ce8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc4207ec418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc4207ec418, 0xc42161c000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc4207ec400, 0xc42161c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc4207ec400, 0xc42161c000, 0x1000, 0x1000, 0xc4200505d8, 0x1, 0xc420050608) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc42020e010, 0xc42161c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc421432cc0, 0xc4219fc690, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc421432cc0, 0xc4219fc690, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc421432cc0, 0xc4219fc690, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421920000, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421920000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 106 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421920000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 107 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421920000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 108 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421474000, 0xc42161d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).fill(0xc4218b9140) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).ReadByte(0xc4218b9140, 0x101000000000016, 0x0, 0x2d) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421ab53e0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420cf8a80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4218b91a0, 0x16e4b60, 0xc420cf8ae0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4218b91a0, 0x16e4b60, 0xc420cf8ae0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421587200, 0x16e4b60, 0xc420cf8ae0, 0xc420cf8ae0, 0xc420e0e8e8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421587200, 0xc420cf8ae0, 0x2d, 0xc421f3c9c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequestHeader(0xc420e0e8c0, 0x1d72340, 0xc421587200, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequest(0xc420e0e8c0, 0x1d72340, 0xc421587200, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc420e0e8c0, 0x1d72340, 0xc421587200, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc421474000, 0xc420e0e8c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 180 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4209a0000, 0xc421df3d10, 0x30, 0xf9866a) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4209a0000, 0x1c61988, 0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42000e840, 0xc420759ae0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42000e840, 0xc420759ae0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 181 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253610, 0x72, 0xc42115fce8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc42112e018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc42112e018, 0xc4208bc000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc42112e000, 0xc4208bc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc42112e000, 0xc4208bc000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc42000e840, 0xc4208bc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc4211b3e00, 0xc4217a4a80, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc4211b3e00, 0xc4217a4a80, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc4211b3e00, 0xc4217a4a80, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4209a0000, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4209a0000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 182 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4209a0000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 183 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4209a0000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 184 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421507da0, 0xc421223700, 0x20, 0x20, 0x169d4e0, 0xc4205748c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420d3a990, 0x1d718c0, 0xc421507da0, 0x0, 0xc421507d00) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202a8000, 0xc421223950, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421493da0, 0xc420d22000, 0xc420d3a720, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.call(0xc42056a780, 0xc42000ecc8, 0x13, 0x1bda9a0, 0x4, 0xc421223d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.Call(0xc42056a780, 0xc42000ecc8, 0x13, 0xc421223d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*service).call(0xc42096ab80, 0xc420759b80, 0xc4217a54e8, 0x0, 0xc42071ba00, 0xc420574fc0, 0x19758e0, 0xc420d22000, 0x16, 0x169e020, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc420759b80, 0x1d72340, 0xc42096ac80, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc42044e3c0, 0xc420759b80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 187 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253880, 0x72, 0xc420050658) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc42187c218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc42187c218, 0xc422361d00, 0x1, 0x1) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc42187c200, 0xc422361d81, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc42187c200, 0xc422361d81, 0x1, 0x1, 0xc, 0xc, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc420206628, 0xc422361d81, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.(*connReader).backgroundRead(0xc422361d70) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 109 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253af0, 0x72, 0xc420051e58) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc42187c198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc42187c198, 0xc422c45500, 0x1, 0x1) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc42187c180, 0xc422c45571, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc42187c180, 0xc422c45571, 0x1, 0x1, 0xc4202de420, 0xf9896a, 0x1) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc4202065e8, 0xc422c45571, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/http.(*connReader).backgroundRead(0xc422c45560) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 110 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421bfc0b0, 0xc42115bd10, 0x30, 0xf9866a) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421bfc0b0, 0x1c61988, 0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42020ef78, 0xc422c9cc80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42020ef78, 0xc422c9cc80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 111 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253a20, 0x72, 0xc421df1ce8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc42180c198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc42180c198, 0xc422312000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc42180c180, 0xc422312000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc42180c180, 0xc422312000, 0x1000, 0x1000, 0xc421df1dd8, 0x1, 0xc421df1e08) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc42020ef78, 0xc422312000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc4214a2360, 0xc42099d630, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc4214a2360, 0xc42099d630, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc4214a2360, 0xc42099d630, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421bfc0b0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421bfc0b0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 112 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421bfc0b0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 113 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421bfc0b0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 195 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b2532d0, 0x72, 0xc42115dce8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc422445b98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc422445b98, 0xc4224ad000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc422445b80, 0xc4224ad000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc422445b80, 0xc4224ad000, 0x1000, 0x1000, 0xc4213995d8, 0x1, 0xc421399608) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc4204a3380, 0xc4224ad000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc4216c2c60, 0xc421d7b830, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc4216c2c60, 0xc421d7b830, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc4216c2c60, 0xc421d7b830, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421bfc160, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421bfc160) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 65 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42179ba20) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 216 [runnable]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldBlock(0xc421e29b80, 0x1a1c000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6607 +0x2a4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4202a8000, 0xc4202c6d01, 0xc4202c6d00) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:300 +0x152 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4202a8000, 0xc421fb72c0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4202a8000, 0xc421fb72c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420c04030, 0xc4202a8000, 0xc421fb72c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 200 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4205da000, 0xc42044f500) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:123 +0x1fd Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205da000, 0xc4217663e0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 189 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4221ec160, 0xc420ee0d10, 0x30, 0xf9866a) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4221ec160, 0x1c61988, 0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42000ed28, 0xc4226664b0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc42000ed28, 0xc4226664b0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 231 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4221ec000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 230 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253470, 0x72, 0xc422b13ce8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc422116118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc422116118, 0xc421e58000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc422116100, 0xc421e58000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc422116100, 0xc421e58000, 0x1000, 0x1000, 0xc422b13dd8, 0x0, 0xc422b13e08) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc4204a3030, 0xc421e58000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc4211a4840, 0xc421e10370, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc4211a4840, 0xc421e10370, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc4211a4840, 0xc421e10370, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4221ec000, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4221ec000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 232 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4221ec000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 233 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b2533a0, 0x72, 0xc42051ace8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc422116298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc422116298, 0xc42255b000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc422116280, 0xc42255b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc422116280, 0xc42255b000, 0x1000, 0x1000, 0xc42051add8, 0x1, 0xc42051ae08) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc42000ed28, 0xc42255b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc4211a4ea0, 0xc4224f87d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc4211a4ea0, 0xc4224f87d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc4211a4ea0, 0xc4224f87d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4221ec160, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4221ec160) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 234 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4221ec160) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 235 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4221ec160) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 236 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421474a80, 0xc422814000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).fill(0xc4215a0e40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).ReadByte(0xc4215a0e40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421f36360, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421366520) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4215a0ea0, 0x16e4b60, 0xc4213665a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4215a0ea0, 0x16e4b60, 0xc4213665a0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422706980, 0x16e4b60, 0xc4213665a0, 0xc4213665a0, 0xc42221f518) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422706980, 0xc4213665a0, 0x2d, 0xc4224e43e0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequestHeader(0xc42221f4f0, 0x1d72340, 0xc422706980, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420201590, 0x8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequest(0xc42221f4f0, 0x1d72340, 0xc422706980, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc42221f4f0, 0x1d72340, 0xc422706980, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc421474a80, 0xc42221f4f0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 237 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421474b40, 0xc4228dc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).fill(0xc421b19740) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).ReadByte(0xc421b19740, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4221610e0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421582400) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421b197a0, 0x16e4b60, 0xc421582460) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421b197a0, 0x16e4b60, 0xc421582460, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422620c00, 0x16e4b60, 0xc421582460, 0xc421582460, 0xc42221f518) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422620c00, 0xc421582460, 0x2d, 0xc422492380) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequestHeader(0xc42221f4f0, 0x1d72340, 0xc422620c00, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420201590, 0x8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequest(0xc42221f4f0, 0x1d72340, 0xc422620c00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc42221f4f0, 0x1d72340, 0xc422620c00, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc421474b40, 0xc42221f4f0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 238 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421bfc160, 0xc420065d10, 0x30, 0xf9866a) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421bfc160, 0x1c61988, 0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc4204a3380, 0xc42221f6d0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc4204a3380, 0xc42221f6d0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 196 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421bfc160) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 197 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421bfc160) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 198 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42018b980, 0xc422ad0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).fill(0xc422323500) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).ReadByte(0xc422323500, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4221872f0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42002d660) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4223235c0, 0x16e4b60, 0xc42002d6c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4223235c0, 0x16e4b60, 0xc42002d6c0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42235fc80, 0x16e4b60, 0xc42002d6c0, 0xc42002d6c0, 0xc422c9d018) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42235fc80, 0xc42002d6c0, 0x2d, 0xc421de3b00) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequestHeader(0xc422c9cff0, 0x1d72340, 0xc42235fc80, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420201590, 0x8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequest(0xc422c9cff0, 0x1d72340, 0xc42235fc80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc422c9cff0, 0x1d72340, 0xc42235fc80, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc42018b980, 0xc422c9cff0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 239 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4221ec210, 0xc421deed10, 0x30, 0xf9866a) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4221ec210, 0x1c61988, 0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc4204a3388, 0xc42221f720) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc4204a3388, 0xc42221f720) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 240 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253200, 0x72, 0xc4213a0ce8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc422445c18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc422445c18, 0xc422734000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc422445c00, 0xc422734000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc422445c00, 0xc422734000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc4204a3388, 0xc422734000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc4222be300, 0xc4226a5670, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc4222be300, 0xc4226a5670, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc4222be300, 0xc4226a5670, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4221ec210, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4221ec210) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 241 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4221ec210) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 242 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4221ec210) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 243 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422702420, 0xc422945700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422226450, 0x1d718c0, 0xc422702420, 0x0, 0xc422702400) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4202a8000, 0xc422945950, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4224ea300, 0xc42230a580, 0xc422226360, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.call(0xc42056a780, 0xc4204a3790, 0x13, 0x1bda9a0, 0x4, 0xc422945d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: reflect.Value.Call(0xc42056a780, 0xc4204a3790, 0x13, 0xc422945d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*service).call(0xc422621a00, 0xc42221f7c0, 0xc421e7a1d8, 0x0, 0xc4217da500, 0xc421582ea0, 0x19758e0, 0xc42230a580, 0x16, 0x169e020, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc42221f7c0, 0x1d72340, 0xc422621b40, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc4209f2cc0, 0xc42221f7c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 202 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4221ec2c0, 0xc42212bd10, 0x30, 0xf9866a) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4221ec2c0, 0x1c61988, 0xc4202a8000, 0x1d71840, 0xc420fafd40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc4204a3858, 0xc42221fbd0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d320, 0xc4204a3858, 0xc42221fbd0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 201 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4205da000, 0xc42044f5c0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4205da000, 0xc42044f5c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205da000, 0xc421766400) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 193 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4205da000, 0xc42044f500, 0xc42247cf60) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205da000, 0xc421583a60) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 259 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4205da000, 0xc42044f5c0, 0xc42247d0e0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205da000, 0xc421583b00) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 203 [IO wait]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.runtime_pollWait(0x7fe55b253060, 0x72, 0xc421decce8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).wait(0xc422116598, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*pollDesc).waitRead(0xc422116598, 0xc422563000, 0x1000, 0x1000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: internal/poll.(*FD).Read(0xc422116580, 0xc422563000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*netFD).Read(0xc422116580, 0xc422563000, 0x1000, 0x1000, 0xc421decdd8, 0x1, 0xc421dece08) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net.(*conn).Read(0xc4204a3858, 0xc422563000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).Read(0xc421c966c0, 0xc421e117e4, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadAtLeast(0x1d59aa0, 0xc421c966c0, 0xc421e117e4, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: io.ReadFull(0x1d59aa0, 0xc421c966c0, 0xc421e117e4, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4221ec2c0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4221ec2c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 204 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4221ec2c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 205 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4221ec2c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 267 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420743f40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 261 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4209f3440, 0xc4220de000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).fill(0xc42259bc80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).ReadByte(0xc42259bc80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4221dd980, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4219b8100) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42259bce0, 0x16e4b60, 0xc4219b8160) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42259bce0, 0x16e4b60, 0xc4219b8160, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4227076c0, 0x16e4b60, 0xc4219b8160, 0xc4219b8160, 0xc42221f7e8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4227076c0, 0xc4219b8160, 0x2d, 0xc422492840) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequestHeader(0xc42221f7c0, 0x1d72340, 0xc4227076c0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420201590, 0x8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequest(0xc42221f7c0, 0x1d72340, 0xc4227076c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc42221f7c0, 0x1d72340, 0xc4227076c0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc4209f3440, 0xc42221f7c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 208 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42044f680, 0xc422ca2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).fill(0xc421507320) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: bufio.(*Reader).ReadByte(0xc421507320, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42220c7b0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421766e40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421507380, 0x16e4b60, 0xc421766ea0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421507380, 0x16e4b60, 0xc421766ea0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42203edc0, 0x16e4b60, 0xc421766ea0, 0xc421766ea0, 0xc42221fc98) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42203edc0, 0xc421766ea0, 0x2d, 0xc421f98fc0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequestHeader(0xc42221fc70, 0x1d72340, 0xc42203edc0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420201590, 0x8) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).readRequest(0xc42221fc70, 0x1d72340, 0xc42203edc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: net/rpc.(*Server).ServeRequest(0xc42221fc70, 0x1d72340, 0xc42203edc0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4202a8000, 0x1d71840, 0xc420fafd40, 0x1d7d0e0, 0xc42044f680, 0xc42221fc70) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 253 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4223bf280) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 254 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4205da000, 0xc42044f5c0, 0x1d6eb80, 0xc4223bf280, 0xc4228d8d20, 0xc4228d8d80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205da000, 0xc4223bf2c0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 268 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420743f40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 269 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc420300be0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4202a8000) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 271 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc42018e120) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 272 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc42018e120) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 273 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc4223041b0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 290 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422263a00, 0xc422996ce8, 0x20, 0x20, 0x18, 0xc421c7a2a0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4222eaf90, 0x1d71840, 0xc422263a00, 0xc421c7a2a0, 0x184987) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4202c6d00, 0xc422941ed0, 0x184987, 0x1d71840, 0xc422263a00, 0xc420cdc260, 0xc422304180, 0xc420fe6300, 0x24, 0x2bba400) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420cdc200, 0x1d71840, 0xc422263a00, 0x184987, 0xc4223bf440, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420cdc200, 0x1d71840, 0xc422263a00) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 291 [runnable]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: sync.(*Mutex).Unlock(0xc420300960) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/sync/mutex.go:175 +0xd5 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: log.(*Logger).Output(0xc420300960, 0x2, 0xc420f6f2c0, 0x4e, 0x0, 0x0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/log/log.go:173 +0x218 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: log.(*Logger).Printf(0xc420300960, 0x1c3f9d0, 0x47, 0xc422985d38, 0x3, 0x3) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /usr/local/go/src/log/log.go:179 +0x7e Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc420130070) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:146 +0xe7a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 292 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422263b00, 0xc422983a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422305080, 0x1d71840, 0xc422263b00, 0xc422305200, 0x1833a4) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4202c6d00, 0xc422983c68, 0x1833a4, 0x1d71840, 0xc422263b00, 0x552bde, 0xc420300960, 0x2, 0xc4213badc0, 0x40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422263c40, 0x1833a4, 0x3b, 0xc422b0fde8, 0x1, 0x1) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422263c40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 293 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422263c80) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 294 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420fca780, 0x1d71840, 0xc422263b00) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 255 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4222eacf0, 0xc4228d8de0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 256 [select]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4222eacf0, 0xc4228d8e40) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: goroutine 257 [runnable]: Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc422c4e1e0) Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:27:03 ip-172-24-20-236 nomad.sh[25530]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:27:03 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:27:03 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:27:03 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: ==> Starting Nomad agent... Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: ==> Nomad agent configuration: Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: Client: false Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: Log Level: INFO Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: Region: amrv01 (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: Server: true Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: Version: 0.8.4 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.230155 [INFO] nomad: starting 4 scheduling worker(s) for [system service batch _core] Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: Attempting re-join to previously known node: consulnomad-staging-03.asv01: 172.27.0.64:4648 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.230246 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [WARN] memberlist: Refuting an alive message Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.584406 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.584431 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.584728 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.584749 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.584761 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.584774 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.584786 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.584815 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.584827 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.584839 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.585234 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.585250 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.585262 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.585273 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.585284 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.585311 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.585323 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.585335 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04.585359 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:04 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:04 [INFO] serf: Re-joined to previously known node: consulnomad-staging-03.asv01: 172.27.0.64:4648 Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:05 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:05 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:06 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:06.427506 [ERR] http: Request /v1/jobs?prefix=elenevskiy-a-jupiternoteBook®ion=amrv01, error: rpc error: stream closed Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 24 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07.433385 [INFO] nomad: cluster leadership acquired Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: 2018/08/23 15:27:07 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: fatal error: concurrent map read and map write Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 300 [running]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc4229b7d20 sp=0xc4229b7d00 pc=0x42d161 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: runtime.mapaccess2(0x1844be0, 0xc4202fe090, 0xc4229b7de0, 0xc422640118, 0x8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc4229b7d68 sp=0xc4229b7d20 pc=0x409c75 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc420210200, 0x1d6dd40, 0xc422640118, 0x0, 0xed3109f1b) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:156 +0xdc fp=0xc4229b7e50 sp=0xc4229b7d68 pc=0xf215ac Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc420488140, 0x1d71840, 0xc42232e800) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc4229b7fc8 sp=0xc4229b7e50 pc=0xf56bb7 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: runtime.goexit() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc4229b7fd0 sp=0xc4229b7fc8 pc=0x45c711 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 1 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc4203577a0, 0xc42049d7c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc4203577a0, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc42031b7c0, 0xc42031b7c0, 0xc420143440, 0x37) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204ddf68) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: main.main() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 5 [syscall]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: os/signal.signal_recv(0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: os/signal.loop() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by os/signal.init.0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 36 [select, locked to thread]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: runtime.selectgo(0xc420267f50, 0xc4204f0120) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: runtime.ensureSigM.func1() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: runtime.goexit() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 131 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4201be000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4201be000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4201be000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4201be000, 0xc4201fc440) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 173 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420506480, 0xc42148a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).fill(0xc4214ddf20) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).ReadByte(0xc4214ddf20, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421957c20, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42105b680) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4217a6000, 0x16e4b60, 0xc42105b6e0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4217a6000, 0x16e4b60, 0xc42105b6e0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421a556c0, 0x16e4b60, 0xc42105b6e0, 0xc42105b6e0, 0xc42058a0c8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421a556c0, 0xc42105b6e0, 0x2d, 0xc420dfbbe0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequestHeader(0xc42058a0a0, 0x1d72340, 0xc421a556c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420208910, 0x8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequest(0xc42058a0a0, 0x1d72340, 0xc421a556c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc42058a0a0, 0x1d72340, 0xc421a556c0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc420506480, 0xc42058a0a0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 95 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/http.(*persistConn).writeLoop(0xc420cc5b00) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by net/http.(*Transport).dialConn Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 107 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4200de780, 0x1bdd09a, 0x6, 0xc421335a40) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 57 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc4204008c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 58 [sleep]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: time.Sleep(0x3b9aca00) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420384790) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 59 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc42044a300) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 24 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc420072080, 0x1d71840, 0xc420082040) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 60 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc420086540) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 133 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4201be000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4201be000, 0xc4201fc460) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 134 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc4213358f0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 62 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc420498270) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0xd40010) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4202fe870, 0xc42036a300) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 136 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2d60, 0x72, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc420564018, 0x72, 0xc420083700, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc420564018, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Accept(0xc420564000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).accept(0xc420564000, 0xc42000e900, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*TCPListener).accept(0xc4204bc1d0, 0x458e40, 0xc420519790, 0xc420519798) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*TCPListener).AcceptTCP(0xc4204bc1d0, 0x1c63658, 0xc420dc63c0, 0xc42000e900) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420dc63c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 42 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2f00, 0x72, 0xc4200659a8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc420452318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc420452318, 0xc420478000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc420452300, 0xc420478000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc420452300, 0xc420478000, 0x1000, 0x1000, 0x458600, 0xc42104f500, 0x4) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc42020e048, 0xc420478000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/http.(*persistConn).Read(0xc420288120, 0xc420478000, 0x1000, 0x1000, 0xc420065b98, 0x405675, 0xc4204f0660) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).fill(0xc4201f66c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Peek(0xc4201f66c0, 0x1, 0x0, 0x0, 0x1, 0xc420d60d80, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/http.(*persistConn).readLoop(0xc420288120) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by net/http.(*Transport).dialConn Aug 23 15:27:07 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 43 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/http.(*persistConn).writeLoop(0xc420288120) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by net/http.(*Transport).dialConn Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 63 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4200a9540, 0x1c636a0, 0xc4203848f0, 0x1d71840, 0xc4201ffa80) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4203848f0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 103 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420dc63c0, 0xc420df2060) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 94 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2a20, 0x72, 0xc4200149a8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc420217d98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc420217d98, 0xc421526000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc420217d80, 0xc421526000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc420217d80, 0xc421526000, 0x1000, 0x1000, 0x458600, 0xc4201ea600, 0x4) Aug 23 15:27:07 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc42020e1f8, 0xc421526000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/http.(*persistConn).Read(0xc420cc5b00, 0xc421526000, 0x1000, 0x1000, 0xc420014b98, 0x405675, 0xc420e6cae0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).fill(0xc421331500) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Peek(0xc421331500, 0x1, 0x0, 0x0, 0x1, 0xc420ea59e0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/http.(*persistConn).readLoop(0xc420cc5b00) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by net/http.(*Transport).dialConn Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 102 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420dc63c0, 0x12a05f200, 0xc42134aba0, 0xc420df2060, 0xc4201fc580) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:110 +0xef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 202 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a23a0, 0x72, 0xc421e2bce8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc42206a018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc42206a018, 0xc42148f000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc42206a000, 0xc42148f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc42206a000, 0xc42148f000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc4204bc048, 0xc42148f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Read(0xc420e10c60, 0xc4226cc000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadAtLeast(0x1d59aa0, 0xc420e10c60, 0xc4226cc000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadFull(0x1d59aa0, 0xc420e10c60, 0xc4226cc000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4226d8000, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4226d8000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 177 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4219693f0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 105 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4200de780) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 135 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420602270) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 104 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420dc63c0, 0x1dcd6500, 0xc42134ac00, 0xc420df2060, 0xc4201fc590) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 132 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4201be000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4201be000, 0xc4201fc450) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 201 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4226d8000, 0xc420fbed10, 0x30, 0xf9866a) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4226d8000, 0x1c61988, 0xc42048b680, 0x1d71840, 0xc421322200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc4204bc048, 0xc42240c230) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc4204bc048, 0xc42240c230) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 106 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4200de780) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 236 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421a88580) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 137 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2c90, 0x72, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc420987698, 0x72, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc420987698, 0xc422fa4000, 0x10000, 0x10000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).ReadFrom(0xc420987680, 0xc422fa4000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).readFrom(0xc420987680, 0xc422fa4000, 0x10000, 0x10000, 0xeb, 0xeb, 0x1d62b40, 0x412539, 0xc421420e40) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*UDPConn).readFrom(0xc4204bc1f0, 0xc422fa4000, 0x10000, 0x10000, 0x101, 0xc421420e18, 0xc421420e90, 0x443cf7) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*UDPConn).ReadFrom(0xc4204bc1f0, 0xc422fa4000, 0x10000, 0x10000, 0x10000, 0x10000, 0x6af7, 0xa39905cd, 0x2b96de0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420dc63c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 138 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420dc63c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 108 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4200de780, 0x1bdba6c, 0x5, 0xc421335a70) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 109 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4200de780, 0x1bdbc6a, 0x5, 0xc421335aa0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 225 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4201be000, 0xc4220ca540, 0x185fcd, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:171 +0x7a2 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4201be000, 0xc4220ca540) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:135 +0x258 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4201be000, 0xc421429780) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 111 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc42134ac60, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc42134ac60, 0x1dcd6500, 0xc420366a40, 0xc420366a60, 0xc420366a80, 0xc420366aa0, 0xc420366ac0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc42134ac60) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 112 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc42134acc0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc42134acc0, 0x1dcd6500, 0xc4203392c0, 0xc4203392f0, 0xc4203393b0, 0xc420339350, 0xc420339380) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc42134acc0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 113 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc42134ad20, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc42134ad20, 0x1dcd6500, 0x2, 0x2, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc42134ad20) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 146 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc42134ad80, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc42134ad80, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc42134ad80) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 147 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a27b0, 0x72, 0xc421391490) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc4220ee718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc4220ee718, 0xc421d8b000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc4220ee700, 0xc421d8b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc4220ee700, 0xc421d8b000, 0x1000, 0x1000, 0x1, 0xc421d8b000, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc42020ed70, 0xc421d8b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Read(0xc42224dda0, 0xc42296b244, 0x1, 0x1, 0x1000, 0x1000, 0xc421391698) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).readTCP(0xc420dc63c0, 0x1d7d320, 0xc42020ed70, 0x2b96de0, 0x0, 0x0, 0x2b96de0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:912 +0xee Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc420dc63c0, 0xc422a03380, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:772 +0x357 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc420dc63c0, 0xc422a03380, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc420dc63c0, 0xc420446800, 0x13, 0x20, 0x3, 0x3, 0xc420c48380) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc4200de780, 0xc420446800, 0x13, 0x20, 0xc420c48301, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc42048b680, 0xc420446800, 0x13, 0x20, 0xc420c48380, 0x11, 0x10) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc420263740, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc4214cf9b0, 0xc42048b680) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 148 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc42048b680) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 149 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc42048b680) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 150 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2e30, 0x72, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc420217798, 0x72, 0xc420082300, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc420217798, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Accept(0xc420217780, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).accept(0xc420217780, 0x1, 0x1, 0xc4209fcf20) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*TCPListener).accept(0xc4204bcd88, 0x406b4a, 0xc420eba4e0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*TCPListener).Accept(0xc4204bcd88, 0xc420eba4e0, 0x0, 0x3f800000, 0xc421322200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc42048b680, 0x1d71840, 0xc421322200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 151 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc420488140, 0x3b9aca00, 0xc4202ee480) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 152 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4200a9310, 0x3b9aca00, 0xc4202ee480) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 153 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420256480, 0x3b9aca00, 0xc4202ee480) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 154 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc420498270, 0x3b9aca00, 0xc4202ee480) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 155 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc42048b680) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 156 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2bc0, 0x72, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc420e28698, 0x72, 0xc420083600, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc420e28698, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Accept(0xc420e28680, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).accept(0xc420e28680, 0x402037, 0xc421628080, 0x1805ec0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*TCPListener).accept(0xc4204bca18, 0x458e40, 0xc42141ee90, 0xc42141ee98) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*TCPListener).Accept(0xc4204bca18, 0x1c65af8, 0xc421628000, 0x1d71900, 0xc421402240) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/http.(*Server).Serve(0xc42059e680, 0x1d6fc00, 0xc4204bca18, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/http.Serve(0x1d6fc00, 0xc4204bca18, 0x1d5df00, 0xc421402180, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc4213222c0, 0x1d6fc00, 0xc4204bca18, 0xc4201fcce0, 0xc4214cfa70) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 157 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420e28880, 0xc4201fccf0, 0xc420df2180) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 158 [sleep]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: time.Sleep(0x1b093c57f) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc4203577a0, 0xc420e28880) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 210 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420384840, 0xc4209f6d10, 0x30, 0xf9866a) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420384840, 0x1c61988, 0xc42048b680, 0x1d71840, 0xc421322200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc42020e028, 0xc42103dc20) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc42020e028, 0xc42103dc20) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 191 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421f0f080) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 211 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2540, 0x72, 0xc420fc0ce8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc4213bc018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc4213bc018, 0xc420c84000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc4213bc000, 0xc420c84000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc4213bc000, 0xc420c84000, 0x1000, 0x1000, 0xc420e66dd8, 0x1, 0xc420e66e08) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc42020e028, 0xc420c84000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Read(0xc42038baa0, 0xc420194560, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadAtLeast(0x1d59aa0, 0xc42038baa0, 0xc420194560, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadFull(0x1d59aa0, 0xc42038baa0, 0xc420194560, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420384840, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420384840) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 192 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421f0f080) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 176 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4219693f0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 190 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2610, 0x72, 0xc4209fdce8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc4220ec018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc4220ec018, 0xc4223d6000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc4220ec000, 0xc4223d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc4220ec000, 0xc4223d6000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc42000e4c8, 0xc4223d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Read(0xc4217a7b00, 0xc4221eb610, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadAtLeast(0x1d59aa0, 0xc4217a7b00, 0xc4221eb610, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadFull(0x1d59aa0, 0xc4217a7b00, 0xc4221eb610, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421f0f080, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421f0f080) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 212 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420384840) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 213 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420384840) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 214 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4214ec000, 0xc420b1c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).fill(0xc42140e120) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).ReadByte(0xc42140e120, 0x101000000000016, 0x0, 0x2d) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420855b60, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420674d60) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42140e180, 0x16e4b60, 0xc420674dc0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42140e180, 0x16e4b60, 0xc420674dc0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e2ab40, 0x16e4b60, 0xc420674dc0, 0xc420674dc0, 0xc42103dce8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e2ab40, 0xc420674dc0, 0x2d, 0xc420546d20) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequestHeader(0xc42103dcc0, 0x1d72340, 0xc420e2ab40, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420208910, 0x8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequest(0xc42103dcc0, 0x1d72340, 0xc420e2ab40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc42103dcc0, 0x1d72340, 0xc420e2ab40, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc4214ec000, 0xc42103dcc0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 215 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4203849a0, 0xc420fc4d10, 0x30, 0xf9866a) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4203849a0, 0x1c61988, 0xc42048b680, 0x1d71840, 0xc421322200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc4202040e8, 0xc42058a000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc4202040e8, 0xc42058a000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 216 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2af0, 0x72, 0xc420062ce8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc42088c098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc42088c098, 0xc42148e000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc42088c080, 0xc42148e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc42088c080, 0xc42148e000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc4202040e8, 0xc42148e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Read(0xc421442060, 0xc420f6c030, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadAtLeast(0x1d59aa0, 0xc421442060, 0xc420f6c030, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadFull(0x1d59aa0, 0xc421442060, 0xc420f6c030, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4203849a0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4203849a0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 217 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4203849a0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 218 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4203849a0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 219 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4217a6120, 0xc42133f700, 0x20, 0x20, 0x169d4e0, 0xc4214062c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421288690, 0x1d718c0, 0xc4217a6120, 0x0, 0xc4217a6100) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42048b680, 0xc42133f950, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421443ec0, 0xc420c6ce80, 0xc421288510, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: reflect.Value.call(0xc42038a960, 0xc420204bc0, 0x13, 0x1bda9a0, 0x4, 0xc42133fd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: reflect.Value.Call(0xc42038a960, 0xc420204bc0, 0x13, 0xc42133fd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*service).call(0xc421850240, 0xc42058a0a0, 0xc421c26a40, 0x0, 0xc420c80f80, 0xc420dc2120, 0x19758e0, 0xc420c6ce80, 0x16, 0x169e020, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc42058a0a0, 0x1d72340, 0xc421850280, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc421018000, 0xc42058a0a0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 220 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421a40000, 0xc420fc5d10, 0x30, 0xf9866a) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421a40000, 0x1c61988, 0xc42048b680, 0x1d71840, 0xc421322200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc42020e998, 0xc421598500) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc42020e998, 0xc421598500) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 221 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2470, 0x72, 0xc421e2ace8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc421fc2418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc421fc2418, 0xc42105c000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc421fc2400, 0xc42105c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc421fc2400, 0xc42105c000, 0x1000, 0x1000, 0xc4202625d8, 0x1, 0xc420262608) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc42020e998, 0xc42105c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Read(0xc4212a2480, 0xc421e4e650, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadAtLeast(0x1d59aa0, 0xc4212a2480, 0xc421e4e650, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadFull(0x1d59aa0, 0xc4212a2480, 0xc421e4e650, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421a40000, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421a40000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 222 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421a40000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 223 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421a40000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 224 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42044a000, 0xc42105d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).fill(0xc4214dca20) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).ReadByte(0xc4214dca20, 0x101000000000016, 0x0, 0x2d) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421a166c0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206c83c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4214dca80, 0x16e4b60, 0xc4206c8420) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4214dca80, 0x16e4b60, 0xc4206c8420, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42198c280, 0x16e4b60, 0xc4206c8420, 0xc4206c8420, 0xc4215985c8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42198c280, 0xc4206c8420, 0x2d, 0xc421fa04c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequestHeader(0xc4215985a0, 0x1d72340, 0xc42198c280, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420208910, 0x8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequest(0xc4215985a0, 0x1d72340, 0xc42198c280, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc4215985a0, 0x1d72340, 0xc42198c280, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc42044a000, 0xc4215985a0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 203 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4226d8000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 204 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4226d8000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 205 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42218dda0, 0xc42138f700, 0x20, 0x20, 0x169d4e0, 0xc4214062c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4223a85d0, 0x1d718c0, 0xc42218dda0, 0x0, 0xc42218dd00) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42048b680, 0xc42138f950, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42218c2a0, 0xc42270f080, 0xc4223a8480, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: reflect.Value.call(0xc42038a960, 0xc4204bd318, 0x13, 0x1bda9a0, 0x4, 0xc420fc1d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: reflect.Value.Call(0xc42038a960, 0xc4204bd318, 0x13, 0xc420fc1d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*service).call(0xc422760500, 0xc42240c2d0, 0xc421698fb8, 0x0, 0xc42270ea80, 0xc42019dea0, 0x19758e0, 0xc42270f080, 0x16, 0x169e020, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc42240c2d0, 0x1d72340, 0xc422760540, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc42044a0c0, 0xc42240c2d0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 189 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421f0f080, 0xc420fbfd10, 0x30, 0xf9866a) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421f0f080, 0x1c61988, 0xc42048b680, 0x1d71840, 0xc421322200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc42000e4c8, 0xc420732870) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc42000e4c8, 0xc420732870) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 207 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420506fc0, 0xc422a99000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).fill(0xc42218de60) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).ReadByte(0xc42218de60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4223a87e0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4205ed0e0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42218df20, 0x16e4b60, 0xc4205ed140) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42218df20, 0x16e4b60, 0xc4205ed140, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422761900, 0x16e4b60, 0xc4205ed140, 0xc4205ed140, 0xc42240c2f8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422761900, 0xc4205ed140, 0x2d, 0xc421eef780) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequestHeader(0xc42240c2d0, 0x1d72340, 0xc422761900, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420208910, 0x8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequest(0xc42240c2d0, 0x1d72340, 0xc422761900, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc42240c2d0, 0x1d72340, 0xc422761900, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc420506fc0, 0xc42240c2d0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 175 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a26e0, 0x72, 0xc421e28ce8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc42287f798, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc42287f798, 0xc4221a2000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc42287f780, 0xc4221a2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc42287f780, 0xc4221a2000, 0x1000, 0x1000, 0xc421e28dd8, 0x0, 0xc421e28e08) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc420204c98, 0xc4221a2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Read(0xc42190cd20, 0xc4221eb800, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadAtLeast(0x1d59aa0, 0xc42190cd20, 0xc4221eb800, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadFull(0x1d59aa0, 0xc42190cd20, 0xc4221eb800, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4219693f0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4219693f0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 274 [runnable]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4201be000, 0xc4220ca600, 0xc42284a480) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4201be000, 0xc42055cca0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 233 [runnable]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*EvalBroker).enqueueLocked(0xc420488140, 0xc4207926e0, 0xc42063d847, 0x5) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:279 +0xb8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc420488140, 0xc4207926e0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:245 +0x2d6 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc420488140, 0xc4207926e0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc42048b680, 0xc42055c601, 0xc42055c680) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc42048b680, 0xc4220a32c0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc42048b680, 0xc4220a32c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc4213380a0, 0xc42048b680, 0xc4220a32c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 237 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421a88580) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 258 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4201be000, 0xc4220ca600, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4201be000, 0xc4220ca600) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4201be000, 0xc4214297a0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 243 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4201be000, 0xc4220ca540, 0xc422a58240) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4201be000, 0xc42065c500) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 235 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2880, 0x72, 0xc4209f7ce8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc422206798, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc422206798, 0xc420f01000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc422206780, 0xc420f01000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc422206780, 0xc420f01000, 0x1000, 0x1000, 0x8e, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc42000ef10, 0xc420f01000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Read(0xc4214dc120, 0xc422384360, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadAtLeast(0x1d59aa0, 0xc4214dc120, 0xc422384360, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadFull(0x1d59aa0, 0xc4214dc120, 0xc422384360, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421a88580, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421a88580) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 234 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421a88580, 0xc420012d10, 0x30, 0xf9866a) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421a88580, 0x1c61988, 0xc42048b680, 0x1d71840, 0xc421322200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc42000ef10, 0xc422831ea0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc42000ef10, 0xc422831ea0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 281 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421b9d200, 0xc422617700, 0x20, 0x20, 0x169d4e0, 0xc4214062c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42233da70, 0x1d718c0, 0xc421b9d200, 0x0, 0xc421b9d200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42048b680, 0xc422617950, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420f2dd40, 0xc421f53c00, 0xc42233d8c0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: reflect.Value.call(0xc42038a960, 0xc42000e8b8, 0x13, 0x1bda9a0, 0x4, 0xc421e27d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: reflect.Value.Call(0xc42038a960, 0xc42000e8b8, 0x13, 0xc421e27d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*service).call(0xc422562580, 0xc420732910, 0xc42296aa68, 0x0, 0xc42287ec00, 0xc42055de80, 0x19758e0, 0xc421f53c00, 0x16, 0x169e020, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc420732910, 0x1d72340, 0xc4224b7580, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc42044af00, 0xc420732910) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 308 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4201be000, 0xc4220ca600, 0x1d6eb80, 0xc421e343c0, 0xc4220a3920, 0xc4220a3980) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4201be000, 0xc421e34400) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 307 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc421e343c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 253 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42044afc0, 0xc421bcd000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).fill(0xc422321140) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).ReadByte(0xc422321140, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4222d89f0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42065d640) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4223211a0, 0x16e4b60, 0xc42065d6c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4223211a0, 0x16e4b60, 0xc42065d6c0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4223ef680, 0x16e4b60, 0xc42065d6c0, 0xc42065d6c0, 0xc420732938) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4223ef680, 0xc42065d6c0, 0x2d, 0xc4220b5400) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequestHeader(0xc420732910, 0x1d72340, 0xc4223ef680, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420208910, 0x8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequest(0xc420732910, 0x1d72340, 0xc4223ef680, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc420732910, 0x1d72340, 0xc4223ef680, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc42044afc0, 0xc420732910) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 283 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc422d00000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 309 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4224b7b80, 0xc42298cd28, 0x20, 0x20, 0xc4201e8800, 0x1bd9a98) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224f64e0, 0x1d71840, 0xc4224b7b80, 0xc420dc35c0, 0x1849b5) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42055c680, 0xc421341f20, 0x1849b5, 0x1d71840, 0xc4224b7b80, 0x7453a6f35b1800ce, 0x6d6f63a873757461, 0x7453b16574656c70, 0x6373654473757461, 0xa06e6f6974706972) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc422d00000, 0x1849b5, 0x6f46a9ffffc1776b, 0xc264656472617772, 0x617073656d614ea9, 0x69676552a6a06563, 0x3076726d61a66e6f, 0x31) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc422d00000, 0x1849b5, 0xc421b9dc20) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 290 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421018fc0, 0xc4217f3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).fill(0xc4222eee40) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).ReadByte(0xc4222eee40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4224b3140, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4216449e0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4222eeea0, 0x16e4b60, 0xc421644a40) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4222eeea0, 0x16e4b60, 0xc421644a40, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422306e80, 0x16e4b60, 0xc421644a40, 0xc421644a40, 0xc422831f68) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422306e80, 0xc421644a40, 0x2d, 0xc421a73c60) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequestHeader(0xc422831f40, 0x1d72340, 0xc422306e80, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420208910, 0x8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequest(0xc422831f40, 0x1d72340, 0xc422306e80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc422831f40, 0x1d72340, 0xc422306e80, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc421018fc0, 0xc422831f40) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 291 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4225f4160, 0xc4209fbd10, 0x30, 0xf9866a) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4225f4160, 0x1c61988, 0xc42048b680, 0x1d71840, 0xc421322200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc420204ea0, 0xc422261450) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d320, 0xc420204ea0, 0xc422261450) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 292 [IO wait]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.runtime_pollWait(0x7f3b1e4a2200, 0x72, 0xc420011ce8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).wait(0xc4220eec98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*pollDesc).waitRead(0xc4220eec98, 0xc42211d000, 0x1000, 0x1000) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: internal/poll.(*FD).Read(0xc4220eec80, 0xc42211d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*netFD).Read(0xc4220eec80, 0xc42211d000, 0x1000, 0x1000, 0xc420011dd8, 0x1, 0xc420011e08) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net.(*conn).Read(0xc420204ea0, 0xc42211d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).Read(0xc4222ef8c0, 0xc422531440, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadAtLeast(0x1d59aa0, 0xc4222ef8c0, 0xc422531440, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: io.ReadFull(0x1d59aa0, 0xc4222ef8c0, 0xc422531440, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4225f4160, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4225f4160) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 293 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4225f4160) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 294 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4225f4160) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 295 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421f8a2a0, 0xc4223c5700, 0x20, 0x20, 0x169d4e0, 0xc4214062c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4221ddbc0, 0x1d718c0, 0xc421f8a2a0, 0x0, 0xc421f8a200) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42048b680, 0xc4223c5950, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4225a37a0, 0xc422d6af00, 0xc4221dda40, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: reflect.Value.call(0xc42038a960, 0xc4202051a8, 0x13, 0x1bda9a0, 0x4, 0xc42141dd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: reflect.Value.Call(0xc42038a960, 0xc4202051a8, 0x13, 0xc42141dd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*service).call(0xc422307d40, 0xc4222614f0, 0xc422a03df8, 0x0, 0xc4220a5580, 0xc421645360, 0x19758e0, 0xc422d6af00, 0x16, 0x169e020, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc4222614f0, 0x1d72340, 0xc422307ec0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc421019080, 0xc4222614f0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 296 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42044b440, 0xc422a82000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).fill(0xc4220d60c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: bufio.(*Reader).ReadByte(0xc4220d60c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4225082a0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4216454e0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4220d6120, 0x16e4b60, 0xc421645560) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4220d6120, 0x16e4b60, 0xc421645560, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42232e140, 0x16e4b60, 0xc421645560, 0xc421645560, 0xc422261518) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42232e140, 0xc421645560, 0x2d, 0xc4224a0140) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequestHeader(0xc4222614f0, 0x1d72340, 0xc42232e140, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420208910, 0x8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).readRequest(0xc4222614f0, 0x1d72340, 0xc42232e140, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: net/rpc.(*Server).ServeRequest(0xc4222614f0, 0x1d72340, 0xc42232e140, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42048b680, 0x1d71840, 0xc421322200, 0x1d7d0e0, 0xc42044b440, 0xc4222614f0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 297 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4206fef00) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 298 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4206fef00) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 299 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4200a9310, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc42048b680) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 301 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420256480) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 302 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420256480) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 303 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc422509350) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 304 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42232e880, 0xc42298bce8, 0x20, 0x20, 0x18, 0xc420b36780) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224dc690, 0x1d71840, 0xc42232e880, 0xc420b36780, 0x184987) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42055c680, 0xc4227cded0, 0x184987, 0x1d71840, 0xc42232e880, 0xc420987b60, 0xc422509320, 0xc42018b7a0, 0x24, 0x2bba400) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420987b00, 0x1d71840, 0xc42232e880, 0x184987, 0xc4224b7ac0, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420987b00, 0x1d71840, 0xc42232e880) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 305 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42232e900, 0xc4223c7770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224dc810, 0x1d71840, 0xc42232e900, 0x0, 0x1849b5) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42055c680, 0xc4223c7960, 0x1849b5, 0x1d71840, 0xc42232e900, 0x552bde, 0xc4223c7950, 0xc4223c7970, 0xf33ba5, 0xc421a81cc8) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc421a81c70, 0x1d71840, 0xc42232e900, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc421a81c70) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 322 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42232e8c0, 0xc421393a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224f6510, 0x1d71840, 0xc42232e8c0, 0xc4224f6630, 0x1833a4) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42055c680, 0xc421393c68, 0x1833a4, 0x1d71840, 0xc42232e8c0, 0x552bde, 0xc4200a8aa0, 0x2, 0xc4203eae80, 0x40) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc42232e940, 0x1833a4, 0x3b, 0xc421e24de8, 0x1, 0x1) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc42232e940) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 323 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc42232e9c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 324 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420dc65a0, 0x1d71840, 0xc42232e8c0) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 259 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4221ddef0, 0xc422a7c900) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: goroutine 260 [select]: Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4221ddef0, 0xc422a7c960) Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:07 ip-172-24-20-236 nomad.sh[25550]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:07 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:27:07 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:27:07 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: ==> Starting Nomad agent... Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: ==> Nomad agent configuration: Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: Client: false Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: Log Level: INFO Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: Region: amrv01 (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: Server: true Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: Version: 0.8.4 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.213101 [INFO] nomad: starting 4 scheduling worker(s) for [batch system service _core] Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.213230 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: Attempting re-join to previously known node: ip-172-25-5-210.amv01: 172.25.5.210:4648 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.358400 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.358423 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.358435 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.358446 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.358459 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568085 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568107 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568119 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568130 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568156 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568394 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568409 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568420 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568431 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [WARN] memberlist: Refuting an alive message Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568771 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568789 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568800 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568811 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: Re-joined to previously known node: ip-172-25-5-210.amv01: 172.25.5.210:4648 Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08.568861 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:08 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:08 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 25 Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09.301498 [INFO] nomad: cluster leadership acquired Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:09 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:09 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:10 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:10 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:10 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: 2018/08/23 15:27:10 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 1.158160662s Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: fatal error: concurrent map writes Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 212 [running]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc420f87978 sp=0xc420f87958 pc=0x42d161 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: runtime.mapassign(0x1844be0, 0xc4205062a0, 0xc420f87aa8, 0x2bba400) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/hashmap.go:519 +0x56e fp=0xc420f87a08 sp=0xc420f87978 pc=0x40a41e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc4200217a0, 0x1d6dd40, 0xc4200bf618, 0x386bc49b, 0xed310848b, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:99 +0x1d1 fp=0xc420f87af8 sp=0xc420f87a08 pc=0xf20d91 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4200e5b80, 0xc42053a840, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc420f87b60 sp=0xc420f87af8 pc=0xf5388e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4200e5b80, 0xc42053a840) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc420f87b98 sp=0xc420f87b60 pc=0xf534b3 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4204a6a00, 0xc420020001, 0xc420020020) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc420f87c88 sp=0xc420f87b98 pc=0xf77602 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4204a6a00, 0xc420d863c0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc420f87d38 sp=0xc420f87c88 pc=0xf76fff Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4204a6a00, 0xc420d863c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc420f87fa0 sp=0xc420f87d38 pc=0xf76b90 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420abc0a0, 0xc4204a6a00, 0xc420d863c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc420f87fc8 sp=0xc420f87fa0 pc=0xfb8e9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: runtime.goexit() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420f87fd0 sp=0xc420f87fc8 pc=0x45c711 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 1 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42030ce00, 0xc420483180) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42030ce00, 0xc4200aa1a0, 0x1, 0x1, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc420310f00, 0xc420310f00, 0xc420416ca0, 0x37) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: main.RunCustom(0xc4200aa190, 0x2, 0x2, 0xc4204e1f68) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: main.Run(0xc4200aa190, 0x2, 0x2, 0xc42009e058) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: main.main() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 19 [syscall]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: os/signal.signal_recv(0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: os/signal.loop() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by os/signal.init.0 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 185 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4208ba000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 183 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4208ba000, 0xc4212e4d10, 0x30, 0xf9866a) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4208ba000, 0x1c61988, 0xc4204a6a00, 0x1d71840, 0xc420d3d940) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d320, 0xc42000e248, 0xc421247450) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d320, 0xc42000e248, 0xc421247450) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 173 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200da9a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 172 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200da9a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 32 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4203f8e00, 0x1d71840, 0xc420497b40) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 56 [select, locked to thread]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: runtime.selectgo(0xc4204bf750, 0xc4202f4600) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: runtime.ensureSigM.func1() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: runtime.goexit() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 57 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc4203f8d40) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 58 [sleep]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: time.Sleep(0x3b9aca00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc42035ed10) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 59 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420370600) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 60 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc42001b0e0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 89 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420370300, 0xc42132c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).fill(0xc4212cea80) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).ReadByte(0xc4212cea80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4212ec450, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420bf62a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4212ceae0, 0x16e4b60, 0xc420bf6300) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4212ceae0, 0x16e4b60, 0xc420bf6300, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4212dc2c0, 0x16e4b60, 0xc420bf6300, 0xc420bf6300, 0xc420d78258) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4212dc2c0, 0xc420bf6300, 0x2d, 0xc42167f2c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).readRequestHeader(0xc420d78230, 0x1d72340, 0xc4212dc2c0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4203b74a0, 0x8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).readRequest(0xc420d78230, 0x1d72340, 0xc4212dc2c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).ServeRequest(0xc420d78230, 0x1d72340, 0xc4212dc2c0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d0e0, 0xc420370300, 0xc420d78230) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 62 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4203d29c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0xc42011d260, 0xc42011d280) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4205066f0, 0xc4204451e0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 167 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4203f54a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 27 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4203b7b80, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc42035e4d0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:27:10 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 25 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340f00, 0x72, 0xc4200609a8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc42043a218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc42043a218, 0xc4205aa000, 0x1000, 0x1000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).Read(0xc42043a200, 0xc4205aa000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).Read(0xc42043a200, 0xc4205aa000, 0x1000, 0x1000, 0x458600, 0xc4204b8d80, 0x4) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*conn).Read(0xc420560028, 0xc4205aa000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/http.(*persistConn).Read(0xc420566120, 0xc4205aa000, 0x1000, 0x1000, 0xc420060b98, 0x405675, 0xc420562420) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).fill(0xc42054c780) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).Peek(0xc42054c780, 0x1, 0x0, 0x0, 0x1, 0xc420eb94a0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/http.(*persistConn).readLoop(0xc420566120) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by net/http.(*Transport).dialConn Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 129 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/http.(*persistConn).writeLoop(0xc421306000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by net/http.(*Transport).dialConn Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 112 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4202a0840) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:27:10 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a0840, 0xc420068830) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 128 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340950, 0x72, 0xc4212d99a8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc420b5a498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc420b5a498, 0xc421320000, 0x1000, 0x1000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).Read(0xc420b5a480, 0xc421320000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).Read(0xc420b5a480, 0xc421320000, 0x1000, 0x1000, 0x458600, 0xc420b29680, 0x4) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*conn).Read(0xc420560030, 0xc421320000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/http.(*persistConn).Read(0xc421306000, 0xc421320000, 0x1000, 0x1000, 0xc4212d9b98, 0x405675, 0xc420eaaa20) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).fill(0xc420d6f620) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).Peek(0xc420d6f620, 0x1, 0x0, 0x0, 0x1, 0xc420de53e0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/http.(*persistConn).readLoop(0xc421306000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by net/http.(*Transport).dialConn Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 26 [select]: Aug 23 15:27:10 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/http.(*persistConn).writeLoop(0xc420566120) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by net/http.(*Transport).dialConn Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 171 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340880, 0x72, 0xc42132bce8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc4209f6618, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc4209f6618, 0xc420a24000, 0x1000, 0x1000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).Read(0xc4209f6600, 0xc420a24000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).Read(0xc4209f6600, 0xc420a24000, 0x1000, 0x1000, 0xed, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*conn).Read(0xc42000f1b8, 0xc420a24000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).Read(0xc42054db00, 0xc4206b03e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: io.ReadAtLeast(0x1d59aa0, 0xc42054db00, 0xc4206b03e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: io.ReadFull(0x1d59aa0, 0xc42054db00, 0xc4206b03e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200da9a0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200da9a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 165 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340a20, 0x72, 0xc420dddce8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc420d22e18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc420d22e18, 0xc420993000, 0x1000, 0x1000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).Read(0xc420d22e00, 0xc420993000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).Read(0xc420d22e00, 0xc420993000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*conn).Read(0xc4200be920, 0xc420993000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).Read(0xc4200d53e0, 0xc420726bd0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: io.ReadAtLeast(0x1d59aa0, 0xc4200d53e0, 0xc420726bd0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: io.ReadFull(0x1d59aa0, 0xc4200d53e0, 0xc420726bd0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4203f54a0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4203f54a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 168 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42037ede0, 0xc420575700, 0x20, 0x20, 0x169d4e0, 0xc4204de3a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4217c5680, 0x1d718c0, 0xc42037ede0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4204a6a00, 0xc420575950, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420d751a0, 0xc4205fdd80, 0xc420ba9440, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: reflect.Value.call(0xc42058d380, 0xc420222b68, 0x13, 0x1bda9a0, 0x4, 0xc420575d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: reflect.Value.Call(0xc42058d380, 0xc420222b68, 0x13, 0xc420575d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*service).call(0xc4203f9a40, 0xc420564d20, 0xc421384648, 0x0, 0xc4205fcc00, 0xc42072fc80, 0x19758e0, 0xc4205fdd80, 0x16, 0x169e020, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).ServeRequest(0xc420564d20, 0x1d72340, 0xc4203f9a80, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d0e0, 0xc420b1c000, 0xc420564d20) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 90 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42131e000, 0xc421303000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).fill(0xc42100e7e0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).ReadByte(0xc42100e7e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420f02db0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4212c4500) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42100e840, 0x16e4b60, 0xc4212c4560) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42100e840, 0x16e4b60, 0xc4212c4560, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4212b8480, 0x16e4b60, 0xc4212c4560, 0xc4212c4560, 0xc420cf33d8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4212b8480, 0xc4212c4560, 0x2d, 0xc4218452c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).readRequestHeader(0xc420cf33b0, 0x1d72340, 0xc4212b8480, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4203b74a0, 0x8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).readRequest(0xc420cf33b0, 0x1d72340, 0xc4212b8480, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).ServeRequest(0xc420cf33b0, 0x1d72340, 0xc4212b8480, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d0e0, 0xc42131e000, 0xc420cf33b0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 111 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4202a0840) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4202a0840) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4202a0840) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a0840, 0xc4200687a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 166 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4203f54a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 113 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4202a0840) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a0840, 0xc420068850) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 130 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420ba82a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 131 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420efa5b0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 132 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340d60, 0x72, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc42043ac98, 0x72, 0xc420c0a200, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc42043ac98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).Accept(0xc42043ac80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).accept(0xc42043ac80, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*TCPListener).accept(0xc42000e228, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*TCPListener).AcceptTCP(0xc42000e228, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4205f43c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 133 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340c90, 0x72, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc42043ad18, 0x72, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc42043ad18, 0xc421f54000, 0x10000, 0x10000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).ReadFrom(0xc42043ad00, 0xc421f54000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).readFrom(0xc42043ad00, 0xc421f54000, 0x10000, 0x10000, 0xeb, 0xeb, 0x1d62b40, 0x412539, 0xc420f84e40) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*UDPConn).readFrom(0xc42000e268, 0xc421f54000, 0x10000, 0x10000, 0x101, 0xc420f84e18, 0xc420f84e90, 0x443cf7) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*UDPConn).ReadFrom(0xc42000e268, 0xc421f54000, 0x10000, 0x10000, 0x10000, 0x10000, 0x5db9, 0x7d2b4011, 0x2b96de0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4205f43c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 134 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4205f43c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 135 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4205f43c0, 0x12a05f200, 0xc420ba7bc0, 0xc420ece3c0, 0xc420069aa0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 136 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4205f43c0, 0xc420ece3c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 137 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4205f43c0, 0x1dcd6500, 0xc420ba7c20, 0xc420ece3c0, 0xc420069ab0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 138 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4204ab380) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 139 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4204ab380) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 140 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204ab380, 0x1bdd09a, 0x6, 0xc420ba83f0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 141 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204ab380, 0x1bdba6c, 0x5, 0xc420ba8420) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 142 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204ab380, 0x1bdbc6a, 0x5, 0xc420ba8450) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 170 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200da9a0, 0xc420063d10, 0x30, 0xf9866a) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200da9a0, 0x1c61988, 0xc4204a6a00, 0x1d71840, 0xc420d3d940) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d320, 0xc42000f1b8, 0xc420cf32c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d320, 0xc42000f1b8, 0xc420cf32c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 144 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420ba7c80, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ba7c80, 0x1dcd6500, 0x1bd9928, 0x2, 0x1bdab30, 0x4, 0xc42019a400) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ba7c80) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 145 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420ba7ce0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ba7ce0, 0x1dcd6500, 0x63c900, 0x1, 0x0, 0x435248, 0x1c66060) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ba7ce0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 146 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420ba7d40, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ba7d40, 0x1dcd6500, 0x0, 0xc420f86660, 0xa10b06, 0x1, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ba7d40) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 147 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420ba7da0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ba7da0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ba7da0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 148 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb473406e0, 0x77, 0xc420b29680) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc421948298, 0x77, 0xc4212f1000, 0xc4212f1028, 0x436544) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitWrite(0xc421948298, 0xc4212f1000, 0x28, 0x611b26) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).WaitWrite(0xc421948280, 0x1c65988, 0x1d718c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:440 +0x37 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).connect(0xc421948280, 0x1d718c0, 0xc422620d20, 0x0, 0x0, 0x1d5c800, 0xc422853b40, 0x0, 0x0, 0x0, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:152 +0x299 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).dial(0xc421948280, 0x1d718c0, 0xc422620d20, 0x1d7a200, 0x0, 0x1d7a200, 0xc422350f90, 0xc4212f1248, 0x62f2be) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/sock_posix.go:142 +0xe9 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.socket(0x1d718c0, 0xc422620d20, 0x1bda392, 0x3, 0x2, 0x1, 0x0, 0x0, 0x1d7a200, 0x0, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/sock_posix.go:93 +0x1a6 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.internetSocket(0x1d718c0, 0xc422620d20, 0x1bda392, 0x3, 0x1d7a200, 0x0, 0x1d7a200, 0xc422350f90, 0x1, 0x0, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/ipsock_posix.go:141 +0x129 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.doDialTCP(0x1d718c0, 0xc422620d20, 0x1bda392, 0x3, 0x0, 0xc422350f90, 0x2bb8aa0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/tcpsock_posix.go:62 +0xb9 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.dialTCP(0x1d718c0, 0xc422620d20, 0x1bda392, 0x3, 0x0, 0xc422350f90, 0x7836bc80e, 0x2b96de0, 0x6fc238e52) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/tcpsock_posix.go:58 +0xe4 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.dialSingle(0x1d718c0, 0xc422620d20, 0xc421948100, 0x1d62b00, 0xc422350f90, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/dial.go:547 +0x375 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.dialSerial(0x1d718c0, 0xc422620d20, 0xc421948100, 0xc4204d94a0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/dial.go:515 +0x22d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*Dialer).DialContext(0xc4212f1848, 0x1d71880, 0xc4200ba050, 0x1bda392, 0x3, 0xc422853b20, 0x11, 0x0, 0x0, 0x0, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/dial.go:397 +0x678 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*Dialer).Dial(0xc4212f1848, 0x1bda392, 0x3, 0xc422853b20, 0x11, 0x20, 0x20, 0x17503e0, 0x2ade3c8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/dial.go:320 +0x75 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc4205f43c0, 0xc42204e060, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:758 +0x14e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc4205f43c0, 0xc42204e060, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc4205f43c0, 0xc420431a00, 0x13, 0x20, 0x3, 0x3, 0xc420688070) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc4204ab380, 0xc420431a00, 0x13, 0x20, 0xc420688001, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc4204a6a00, 0xc420431a00, 0x13, 0x20, 0xc420688070, 0x10, 0x10) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc420543f40, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc4212c6450, 0xc4204a6a00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 149 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4204a6a00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 150 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4204a6a00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 151 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340e30, 0x72, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc4205c8718, 0x72, 0xc420496d00, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc4205c8718, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).Accept(0xc4205c8700, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).accept(0xc4205c8700, 0x1, 0x1, 0xc4212dbf20) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*TCPListener).accept(0xc42000f360, 0x406b4a, 0xc420f0c120, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*TCPListener).Accept(0xc42000f360, 0xc420f0c120, 0x0, 0x3f800000, 0xc420d3d940) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4204a6a00, 0x1d71840, 0xc420d3d940) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 152 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4200e5b80, 0x3b9aca00, 0xc4202f4a80) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 153 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4203b77c0, 0x3b9aca00, 0xc4202f4a80) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 154 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc42015e630, 0x3b9aca00, 0xc4202f4a80) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 155 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4203d29c0, 0x3b9aca00, 0xc4202f4a80) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 156 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4204a6a00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 157 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340bc0, 0x72, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc420d22118, 0x72, 0xc420083000, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc420d22118, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).Accept(0xc420d22100, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).accept(0xc420d22100, 0x402037, 0xc42138a260, 0x1805ec0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*TCPListener).accept(0xc42000f918, 0x458e40, 0xc4212d5e90, 0xc4212d5e98) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*TCPListener).Accept(0xc42000f918, 0x1c65af8, 0xc42138a1e0, 0x1d71900, 0xc4212ec390) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/http.(*Server).Serve(0xc4203d2000, 0x1d6fc00, 0xc42000f918, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/http.Serve(0x1d6fc00, 0xc42000f918, 0x1d5df00, 0xc4212ec2d0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420d3da00, 0x1d6fc00, 0xc42000f918, 0xc420360040, 0xc4212c6510) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 158 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420d22180, 0xc420360100, 0xc420ecf080) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 28 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4203f5290, 0xc420066d10, 0x30, 0xf9866a) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4203f5290, 0x1c61988, 0xc4204a6a00, 0x1d71840, 0xc420d3d940) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d320, 0xc420222010, 0xc420d78140) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d320, 0xc420222010, 0xc420d78140) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 159 [sleep]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: time.Sleep(0x6d7a45f16) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc42030ce00, 0xc420d22180) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 29 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340af0, 0x72, 0xc4212e0ce8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc4209c8518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc4209c8518, 0xc421302000, 0x1000, 0x1000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).Read(0xc4209c8500, 0xc421302000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).Read(0xc4209c8500, 0xc421302000, 0x1000, 0x1000, 0x2f, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*conn).Read(0xc420222010, 0xc421302000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).Read(0xc420d748a0, 0xc4208101d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: io.ReadAtLeast(0x1d59aa0, 0xc420d748a0, 0xc4208101d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: io.ReadFull(0x1d59aa0, 0xc420d748a0, 0xc4208101d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4203f5290, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4203f5290) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 30 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4203f5290) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 31 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4203f5290) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 195 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4203f54a0, 0xc421327d10, 0x30, 0xf9866a) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4203f54a0, 0x1c61988, 0xc4204a6a00, 0x1d71840, 0xc420d3d940) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d320, 0xc4200be920, 0xc420cf3130) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d320, 0xc4200be920, 0xc420cf3130) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 186 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4208ba000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 198 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4202a0840, 0xc420ee2840, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4202a0840, 0xc420ee2840) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a0840, 0xc4206da4c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 197 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4202a0840, 0xc420ee2780, 0x185fcf, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:171 +0x7a2 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4202a0840, 0xc420ee2780) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:135 +0x258 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a0840, 0xc4206da480) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 184 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340540, 0x72, 0xc420f85ce8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc4209c8098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc4209c8098, 0xc420992000, 0x1000, 0x1000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).Read(0xc4209c8080, 0xc420992000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).Read(0xc4209c8080, 0xc420992000, 0x1000, 0x1000, 0xc4200545d8, 0x1, 0xc420054608) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*conn).Read(0xc42000e248, 0xc420992000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).Read(0xc4200d5740, 0xc420ae8000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: io.ReadAtLeast(0x1d59aa0, 0xc4200d5740, 0xc420ae8000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: io.ReadFull(0x1d59aa0, 0xc4200d5740, 0xc420ae8000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4208ba000, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4208ba000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 213 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4202a0840, 0xc420ee2780, 0xc420d86480) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a0840, 0xc4207004a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 187 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42131e180, 0xc420fc1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).fill(0xc421343aa0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).ReadByte(0xc421343aa0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420ed5bf0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420e4c340) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421343b60, 0x16e4b60, 0xc420e4c3a0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421343b60, 0x16e4b60, 0xc420e4c3a0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420ccf840, 0x16e4b60, 0xc420e4c3a0, 0xc420e4c3a0, 0xc421247518) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420ccf840, 0xc420e4c3a0, 0x2d, 0xc4221fbec0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).readRequestHeader(0xc4212474f0, 0x1d72340, 0xc420ccf840, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4203b74a0, 0x8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).readRequest(0xc4212474f0, 0x1d72340, 0xc420ccf840, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).ServeRequest(0xc4212474f0, 0x1d72340, 0xc420ccf840, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d0e0, 0xc42131e180, 0xc4212474f0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 215 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4202a0840, 0xc420ee2840, 0xc420d869c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a0840, 0xc4207005c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 216 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420d4be00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 217 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4202a0840, 0xc420ee2840, 0x1d6eb80, 0xc420d4be00, 0xc420d871a0, 0xc420d872c0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4202a0840, 0xc420d4be80) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 220 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4208ba0b0, 0xc421329d10, 0x30, 0xf9866a) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4208ba0b0, 0x1c61988, 0xc4204a6a00, 0x1d71840, 0xc420d3d940) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d320, 0xc42000ebb0, 0xc4210f8a00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d320, 0xc42000ebb0, 0xc4210f8a00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 221 [IO wait]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.runtime_pollWait(0x7fcb47340470, 0x72, 0xc420dd8ce8) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).wait(0xc420d16a18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*pollDesc).waitRead(0xc420d16a18, 0xc420d37000, 0x1000, 0x1000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: internal/poll.(*FD).Read(0xc420d16a00, 0xc420d37000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).Read(0xc420d16a00, 0xc420d37000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*conn).Read(0xc42000ebb0, 0xc420d37000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: bufio.(*Reader).Read(0xc42179b200, 0xc4207854a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: io.ReadAtLeast(0x1d59aa0, 0xc42179b200, 0xc4207854a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: io.ReadFull(0x1d59aa0, 0xc42179b200, 0xc4207854a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4208ba0b0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4208ba0b0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 222 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4208ba0b0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 223 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4208ba0b0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 224 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42192e7e0, 0xc4212f3700, 0x20, 0x20, 0x169d4e0, 0xc420fbf480) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4217dbe60, 0x1d718c0, 0xc42192e7e0, 0x0, 0xc42192e700) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4204a6a00, 0xc4212f3950, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4216d9020, 0xc420a2ec00, 0xc4217dbd40, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: reflect.Value.call(0xc42058d380, 0xc42000ff90, 0x13, 0x1bda9a0, 0x4, 0xc4212f3d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: reflect.Value.Call(0xc42058d380, 0xc42000ff90, 0x13, 0xc4212f3d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*service).call(0xc4218150c0, 0xc4210f8aa0, 0xc4207857a8, 0x0, 0xc420a2e400, 0xc4207912e0, 0x19758e0, 0xc420a2ec00, 0x16, 0x169e020, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net/rpc.(*Server).ServeRequest(0xc4210f8aa0, 0x1d72340, 0xc421815140, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a6a00, 0x1d71840, 0xc420d3d940, 0x1d7d0e0, 0xc420b1c0c0, 0xc4210f8aa0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 229 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420f2f040) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 230 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420f2f040) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 97 [runnable]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.(*CompoundIndex).PrefixFromArgs(0xc4205d3060, 0xc422ec4080, 0x2, 0x2, 0xc4205d3060, 0xc421326801, 0xc421326920, 0x2b9c4a0, 0xc421326900) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/index.go:551 +0x48b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.(*Txn).getIndexValue(0xc4223417c0, 0x1bdc14c, 0x5, 0x1be427f, 0x3, 0xc422ec4080, 0x2, 0x2, 0x7fcb4739c6c8, 0x0, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/txn.go:564 +0x1fe Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.(*Txn).Get(0xc4223417c0, 0x1bdc14c, 0x5, 0x1be427f, 0xa, 0xc422ec4080, 0x2, 0x2, 0x20, 0xc422ec4060, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/txn.go:590 +0x9b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/state.(*StateStore).EvalsByJob(0xc421326bb0, 0x0, 0xc420214d94, 0x7, 0xc420214d8c, 0x4, 0x3, 0x0, 0x3, 0x4, ...) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:1821 +0x19d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).latestEvalIndex(0xc422ed2000, 0xc421326ef8, 0xc421326ef8, 0x4) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:793 +0x18b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).handleAllocUpdate(0xc422ed2000, 0xc420560038, 0x1, 0x1, 0x9073f331, 0x2b96de0, 0x4a48ed977d, 0x0, 0xc42053eed0, 0x43e2b2) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:514 +0x40 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc422ed2000) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:441 +0x591 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 204 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: net.(*netFD).connect.func2(0x1d718c0, 0xc422620d20, 0xc421948280, 0xc4221aa1e0, 0xc4221aa180) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:129 +0xde Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by net.(*netFD).connect Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/net/fd_unix.go:128 +0x286 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 231 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4203b77c0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4204a6a00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 232 [semacquire]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: sync.runtime_SemacquireMutex(0xc4200e5c0c, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: sync.(*Mutex).Lock(0xc4200e5c08) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: sync.(*RWMutex).Lock(0xc4200e5c08) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4200e5b80, 0x1d71840, 0xc422e99dc0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 233 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc42015e630) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 234 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc42015e630) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 235 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc422eaf470) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 236 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422e99e00, 0xc421326ce8, 0x20, 0x20, 0x18, 0xc422baa320) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422bb05a0, 0x1d71840, 0xc422e99e00, 0xc422baa320, 0x184987) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420020020, 0xc422edbed0, 0x184987, 0x1d71840, 0xc422e99e00, 0xc420edace0, 0xc422eaf440, 0xc420c67680, 0x24, 0x2bba400) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420edac80, 0x1d71840, 0xc422e99e00, 0x184987, 0xc422b98200, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420edac80, 0x1d71840, 0xc422e99e00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 237 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc420d3b540, 0x7fcb4732e540, 0xc422e99e80, 0x1, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:248 +0x2c1 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc420d3b540, 0x7fcb4732e540, 0xc422e99e80, 0x7fcb4732e540, 0xc422e99e80) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc42282ff80, 0x1d71840, 0xc422e99e80, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:419 +0x7b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc42282ff80) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 238 [runnable]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: sync.(*Mutex).Lock(0xc4203b7540) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/sync/mutex.go:72 +0x2e9 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: log.(*Logger).Output(0xc4203b7540, 0x2, 0xc4205a7e80, 0x7d, 0x0, 0x0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/log/log.go:153 +0x72 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: log.(*Logger).Printf(0xc4203b7540, 0x1c429c3, 0x4e, 0xc42199de48, 0x4, 0x4) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /usr/local/go/src/log/log.go:179 +0x7e Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422e99ec0) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:197 +0x39d Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 239 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422e99f00) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 240 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4205f45a0, 0x1d71840, 0xc422e99e40) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 206 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc42242c210, 0xc42154a960) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: goroutine 207 [select]: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc42242c210, 0xc42154aa80) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25569]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:10 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:27:10 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:27:10 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: ==> Starting Nomad agent... Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: ==> Nomad agent configuration: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: Client: false Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: Log Level: INFO Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: Region: amrv01 (DC: test) Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: Server: true Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: Version: 0.8.4 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:10 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:10 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:10 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:10 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:10.965282 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:10 [INFO] serf: Attempting re-join to previously known node: consulnomad-01.aav01: 172.27.0.55:4648 Aug 23 15:27:10 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:10.965988 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.319844 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.319868 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320021 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320039 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320051 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320063 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320074 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320085 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320095 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [WARN] memberlist: Refuting an alive message Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320471 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320488 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320500 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320514 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320525 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320541 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11 [INFO] serf: Re-joined to previously known node: consulnomad-01.aav01: 172.27.0.55:4648 Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320761 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320773 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320786 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:11 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:11.320797 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:12 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:12 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 27 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14.435355 [INFO] nomad: cluster leadership acquired Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: 2018/08/23 15:27:14 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: fatal error: concurrent map read and map write Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 201 [running]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc4200129c0 sp=0xc4200129a0 pc=0x42d161 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: runtime.mapaccess2(0x1844be0, 0xc42002d4d0, 0xc420012a88, 0x412539, 0xc4203e4518) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc420012a08 sp=0xc4200129c0 pc=0x409c75 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc4203e2660, 0x1d6dd40, 0xc4203e4518, 0x68d3118, 0xed310847f, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:94 +0xe2 fp=0xc420012af8 sp=0xc420012a08 pc=0xf20ca2 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4204e05a0, 0xc420710f20, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc420012b60 sp=0xc420012af8 pc=0xf5388e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4204e05a0, 0xc420710f20) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc420012b98 sp=0xc420012b60 pc=0xf534b3 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc42030cf00, 0xc4203b6501, 0xc4203b65a0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc420012c88 sp=0xc420012b98 pc=0xf77602 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc42030cf00, 0xc422c0a840, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc420012d38 sp=0xc420012c88 pc=0xf76fff Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc42030cf00, 0xc422c0a840) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc420012fa0 sp=0xc420012d38 pc=0xf76b90 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc42050e250, 0xc42030cf00, 0xc422c0a840) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc420012fc8 sp=0xc420012fa0 pc=0xfb8e9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: runtime.goexit() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420012fd0 sp=0xc420012fc8 pc=0x45c711 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 1 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42031a0e0, 0xc420494f00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42031a0e0, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc420236b40, 0xc420236b40, 0xc42020fb60, 0x37) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4201bff68) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: main.main() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 5 [syscall]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: os/signal.signal_recv(0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: os/signal.loop() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by os/signal.init.0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 81 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420a0a3c0, 0x1dcd6500, 0xc420fea6c0, 0xc420f51320, 0xc4203a6560) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 210 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc420022000, 0xc420132240, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc420022000, 0xc420132240) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420022000, 0xc4203af540) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 79 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420a0a3c0, 0x12a05f200, 0xc420fea660, 0xc420f51320, 0xc4203a6550) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 113 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422565ec0, 0xc4229a1700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42248a360, 0x1d718c0, 0xc422565ec0, 0x0, 0xc422565e00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42030cf00, 0xc4229a1950, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc422565560, 0xc421dbb000, 0xc42248a270, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: reflect.Value.call(0xc420591c80, 0xc420202928, 0x13, 0x1bda9a0, 0x4, 0xc4229a1d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: reflect.Value.Call(0xc420591c80, 0xc420202928, 0x13, 0xc4229a1d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*service).call(0xc42253e980, 0xc422d839f0, 0xc422600018, 0x0, 0xc421dbaa80, 0xc422e47940, 0x19758e0, 0xc421dbb000, 0x16, 0x169e020, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc422d839f0, 0x1d72340, 0xc42253e9c0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc4227de0c0, 0xc422d839f0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 9 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420588140, 0x1c636a0, 0xc4200be2c0, 0x1d71840, 0xc420213000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4200be2c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 112 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421d34000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 67 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc420083880, 0x1d71840, 0xc4201fd140) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 45 [select, locked to thread]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: runtime.selectgo(0xc420051750, 0xc4202e69c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: runtime.ensureSigM.func1() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: runtime.goexit() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 46 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc4200837c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 47 [sleep]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: time.Sleep(0x3b9aca00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420235760) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 48 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420132600) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 49 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4200879e0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 51 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4203e88f0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc42002d920, 0xc420396aa0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 27 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646bf00, 0x72, 0xc4200659a8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc42044c918, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc42044c918, 0xc42058e000, 0x1000, 0x1000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Read(0xc42044c900, 0xc42058e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).Read(0xc42044c900, 0xc42058e000, 0x1000, 0x1000, 0x458600, 0xc4204abb00, 0x4) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*conn).Read(0xc4202021b0, 0xc42058e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/http.(*persistConn).Read(0xc4200cc5a0, 0xc42058e000, 0x1000, 0x1000, 0xc420065b98, 0x405675, 0xc42053c480) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).fill(0xc4201f4780) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).Peek(0xc4201f4780, 0x1, 0x0, 0x0, 0x1, 0xc420f20e40, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/http.(*persistConn).readLoop(0xc4200cc5a0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by net/http.(*Transport).dialConn Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 28 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/http.(*persistConn).writeLoop(0xc4200cc5a0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by net/http.(*Transport).dialConn Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 127 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4206f0180, 0xc420d76000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).fill(0xc420ca8ea0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).ReadByte(0xc420ca8ea0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42113d650, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206d2c60) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420ca8f00, 0x16e4b60, 0xc4206d2cc0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420ca8f00, 0x16e4b60, 0xc4206d2cc0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420cbe880, 0x16e4b60, 0xc4206d2cc0, 0xc4206d2cc0, 0xc4211f62a8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420cbe880, 0xc4206d2cc0, 0x2d, 0xc420709940) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequestHeader(0xc4211f6280, 0x1d72340, 0xc420cbe880, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9c20, 0x8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequest(0xc4211f6280, 0x1d72340, 0xc420cbe880, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc4211f6280, 0x1d72340, 0xc420cbe880, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc4206f0180, 0xc4211f6280) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 172 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4225e2210, 0xc4211c9d10, 0x30, 0xf9866a) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4225e2210, 0x1c61988, 0xc42030cf00, 0x1d71840, 0xc420fe2200) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:14 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc42000f9a0, 0xc422d83040) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc42000f9a0, 0xc422d83040) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 110 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646b470, 0x72, 0xc420dcdce8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc421ca0198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc421ca0198, 0xc422223000, 0x1000, 0x1000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Read(0xc421ca0180, 0xc422223000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).Read(0xc421ca0180, 0xc422223000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*conn).Read(0xc420202028, 0xc422223000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).Read(0xc4216035c0, 0xc421ee6b40, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadAtLeast(0x1d59aa0, 0xc4216035c0, 0xc421ee6b40, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadFull(0x1d59aa0, 0xc4216035c0, 0xc421ee6b40, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421d34000, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421d34000) Aug 23 15:27:14 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 111 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421d34000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 130 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc420022000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc420022000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc420022000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420022000, 0xc4203a73b0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 131 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc420022000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420022000, 0xc4203a73c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 80 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420a0a3c0, 0xc420f51320) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 132 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc420022000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420022000, 0xc4203a73e0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 194 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646b880, 0x72, 0xc420dc8ce8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc421f1a198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc421f1a198, 0xc42204d000, 0x1000, 0x1000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Read(0xc421f1a180, 0xc42204d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).Read(0xc421f1a180, 0xc42204d000, 0x1000, 0x1000, 0x17d, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*conn).Read(0xc42000fd20, 0xc42204d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).Read(0xc4219fbf20, 0xc421ee6100, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadAtLeast(0x1d59aa0, 0xc4219fbf20, 0xc421ee6100, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadFull(0x1d59aa0, 0xc4219fbf20, 0xc421ee6100, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4225e22c0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4225e22c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 133 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420fe57d0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 134 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4205f4a90) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 135 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646bd60, 0x72, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc420216118, 0x72, 0xc4203ea100, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc420216118, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Accept(0xc420216100, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).accept(0xc420216100, 0x1afe340, 0xc4204aff58, 0x42bc79) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*TCPListener).accept(0xc4202021a8, 0xc4204aff70, 0xc4204affc8, 0xa17bd6) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*TCPListener).AcceptTCP(0xc4202021a8, 0x4, 0xc4208a8620, 0x15) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420a0a3c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 136 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646bc90, 0x72, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc420216198, 0x72, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc420216198, 0xc422fe8000, 0x10000, 0x10000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).ReadFrom(0xc420216180, 0xc422fe8000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).readFrom(0xc420216180, 0xc422fe8000, 0x10000, 0x10000, 0xe9, 0xe9, 0x1d62b40, 0x412539, 0xc420dcee40) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*UDPConn).readFrom(0xc4202021b8, 0xc422fe8000, 0x10000, 0x10000, 0x101, 0xc420dcee18, 0xc420dcee90, 0x443cf7) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*UDPConn).ReadFrom(0xc4202021b8, 0xc422fe8000, 0x10000, 0x10000, 0x10000, 0x10000, 0xaca7, 0xe504d732, 0x2b96de0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420a0a3c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 137 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420a0a3c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 146 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc42058b380) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 147 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc42058b380) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 148 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42058b380, 0x1bdd09a, 0x6, 0xc420fe5920) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 149 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42058b380, 0x1bdba6c, 0x5, 0xc420fe5950) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 150 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42058b380, 0x1bdbc6a, 0x5, 0xc420fe5980) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 212 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646baf0, 0x72, 0xc422984ce8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc4227d9d98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc4227d9d98, 0xc4227ac000, 0x1000, 0x1000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Read(0xc4227d9d80, 0xc4227ac000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).Read(0xc4227d9d80, 0xc4227ac000, 0x1000, 0x1000, 0xc422984dd8, 0x1, 0xc422984e08) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*conn).Read(0xc42020dd00, 0xc4227ac000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).Read(0xc421a6e660, 0xc421ee66b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadAtLeast(0x1d59aa0, 0xc421a6e660, 0xc421ee66b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadFull(0x1d59aa0, 0xc421a6e660, 0xc421ee66b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42211a160, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42211a160) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 152 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420fea720, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420fea720, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420fea720) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 153 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420fea780, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420fea780, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420fea780) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 154 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420fea7e0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420fea7e0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420fea7e0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 155 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420fea840, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420fea840, 0x1dcd6500, 0x7f07c6482cc0, 0xc4210d86e0, 0x0, 0x0, 0xc420ca2480) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420fea840) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 156 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc42145d980, 0xc42030cf00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 157 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc42030cf00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 158 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc42030cf00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 159 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646be30, 0x72, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc4205ac698, 0x72, 0xc420f8a000, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc4205ac698, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Accept(0xc4205ac680, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).accept(0xc4205ac680, 0x1, 0x1, 0xc420011f20) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*TCPListener).accept(0xc42020d400, 0x406b4a, 0xc420f3d0e0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*TCPListener).Accept(0xc42020d400, 0xc420f3d0e0, 0x0, 0x3f800000, 0xc420fe2200) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc42030cf00, 0x1d71840, 0xc420fe2200) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 160 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4204e05a0, 0x3b9aca00, 0xc4202e6e40) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 161 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4200a9ef0, 0x3b9aca00, 0xc4202e6e40) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 162 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420250990, 0x3b9aca00, 0xc4202e6e40) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 163 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4203e88f0, 0x3b9aca00, 0xc4202e6e40) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 164 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc42030cf00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 165 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646bbc0, 0x72, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc4205aca98, 0x72, 0xc420082000, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc4205aca98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Accept(0xc4205aca80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).accept(0xc4205aca80, 0x412d88, 0x30, 0x1a1fac0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*TCPListener).accept(0xc4202023b8, 0x5fea4a, 0x1a1fac0, 0xc4205946c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*TCPListener).Accept(0xc4202023b8, 0xc42003e028, 0x1830320, 0x2ab0680, 0x1b7b4c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/http.(*Server).Serve(0xc420862750, 0x1d6fc00, 0xc4202023b8, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/http.Serve(0x1d6fc00, 0xc4202023b8, 0x1d5df00, 0xc4205943c0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420fe22c0, 0x1d6fc00, 0xc4202023b8, 0xc4203a6b00, 0xc42145da40) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 166 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc4205acb00, 0xc4203a6b10, 0xc420f518c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 167 [sleep]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: time.Sleep(0x4cb5bc17f) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc42031a0e0, 0xc4205acb00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 213 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42211a160) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 173 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646b950, 0x72, 0xc421242ce8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc4219ab518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc4219ab518, 0xc422524000, 0x1000, 0x1000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Read(0xc4219ab500, 0xc422524000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).Read(0xc4219ab500, 0xc422524000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*conn).Read(0xc42000f9a0, 0xc422524000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).Read(0xc42193b4a0, 0xc422e44a90, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadAtLeast(0x1d59aa0, 0xc42193b4a0, 0xc422e44a90, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadFull(0x1d59aa0, 0xc42193b4a0, 0xc422e44a90, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4225e2210, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4225e2210) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 144 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420234420, 0xc420014d10, 0x30, 0xf9866a) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420234420, 0x1c61988, 0xc42030cf00, 0x1d71840, 0xc420fe2200) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc4202021c0, 0xc421498dc0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc4202021c0, 0xc421498dc0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 145 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646b7b0, 0x72, 0xc4211cbce8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc421422098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc421422098, 0xc42141f000, 0x1000, 0x1000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Read(0xc421422080, 0xc42141f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).Read(0xc421422080, 0xc42141f000, 0x1000, 0x1000, 0xc42004edd8, 0x1, 0xc42004ee08) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*conn).Read(0xc4202021c0, 0xc42141f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).Read(0xc420e037a0, 0xc421213290, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadAtLeast(0x1d59aa0, 0xc420e037a0, 0xc421213290, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadFull(0x1d59aa0, 0xc420e037a0, 0xc421213290, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420234420, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420234420) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 178 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420234420) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 179 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420234420) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 180 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4206f03c0, 0xc4214a2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).fill(0xc42107faa0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).ReadByte(0xc42107faa0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420f6bf50, 0x42c30b) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42116fd40) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42107fb00, 0x16e4b60, 0xc42116fda0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42107fb00, 0x16e4b60, 0xc42116fda0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42117bd80, 0x16e4b60, 0xc42116fda0, 0xc42116fda0, 0xc421498e88) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42117bd80, 0xc42116fda0, 0x2d, 0xc420f2d720) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequestHeader(0xc421498e60, 0x1d72340, 0xc42117bd80, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a9c20, 0x8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequest(0xc421498e60, 0x1d72340, 0xc42117bd80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc421498e60, 0x1d72340, 0xc42117bd80, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc4206f03c0, 0xc421498e60) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 181 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420234840, 0xc4211c6d10, 0x30, 0xf9866a) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420234840, 0x1c61988, 0xc42030cf00, 0x1d71840, 0xc420fe2200) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc42000e008, 0xc4211f61e0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc42000e008, 0xc4211f61e0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 182 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646b6e0, 0x72, 0xc421244ce8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc4203ca018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc4203ca018, 0xc420ffd000, 0x1000, 0x1000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Read(0xc4203ca000, 0xc420ffd000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).Read(0xc4203ca000, 0xc420ffd000, 0x1000, 0x1000, 0xc421244dd8, 0x1, 0xc421244e08) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*conn).Read(0xc42000e008, 0xc420ffd000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).Read(0xc42100c540, 0xc42055a030, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadAtLeast(0x1d59aa0, 0xc42100c540, 0xc42055a030, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadFull(0x1d59aa0, 0xc42100c540, 0xc42055a030, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420234840, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420234840) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 183 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420234840) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 184 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420234840) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 185 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4213c0540, 0xc420fff700, 0x20, 0x20, 0x169d4e0, 0xc4205c4780) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420fb4270, 0x1d718c0, 0xc4213c0540, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42030cf00, 0xc420fff950, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4213f0360, 0xc42142ea00, 0xc420bd7f80, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: reflect.Value.call(0xc420591c80, 0xc42000f568, 0x13, 0x1bda9a0, 0x4, 0xc420fffd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: reflect.Value.Call(0xc420591c80, 0xc42000f568, 0x13, 0xc420fffd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*service).call(0xc420eae700, 0xc4211f6280, 0xc420c56428, 0x0, 0xc420ca6b80, 0xc42100a500, 0x19758e0, 0xc42142ea00, 0x16, 0x169e020, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc4211f6280, 0x1d72340, 0xc420eae740, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc4206f00c0, 0xc4211f6280) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 186 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4220be000, 0xc420016d10, 0x30, 0xf9866a) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4220be000, 0x1c61988, 0xc42030cf00, 0x1d71840, 0xc420fe2200) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc42000e030, 0xc4213f25f0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc42000e030, 0xc4213f25f0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 187 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646b610, 0x72, 0xc42123fce8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc420a7e018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc420a7e018, 0xc421323000, 0x1000, 0x1000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Read(0xc420a7e000, 0xc421323000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).Read(0xc420a7e000, 0xc421323000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*conn).Read(0xc42000e030, 0xc421323000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).Read(0xc421fb8240, 0xc422226c90, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadAtLeast(0x1d59aa0, 0xc421fb8240, 0xc422226c90, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadFull(0x1d59aa0, 0xc421fb8240, 0xc422226c90, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4220be000, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4220be000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 188 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4220be000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 189 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4220be000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 171 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421768180, 0xc420c43700, 0x20, 0x20, 0x169d4e0, 0xc4205c4780) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421eba2a0, 0x1d718c0, 0xc421768180, 0x0, 0xc421768100) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42030cf00, 0xc420c43950, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420ca8120, 0xc420880580, 0xc421eba180, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: reflect.Value.call(0xc420591c80, 0xc42000f8a8, 0x13, 0x1bda9a0, 0x4, 0xc4221efd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: reflect.Value.Call(0xc420591c80, 0xc42000f8a8, 0x13, 0xc4221efd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*service).call(0xc421c31080, 0xc4213f2690, 0xc421ee6000, 0x0, 0xc4209a9680, 0xc421c3a0a0, 0x19758e0, 0xc420880580, 0x16, 0x169e020, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc4213f2690, 0x1d72340, 0xc421f5c000, 0x0, 0xc4204ce000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc4204ce000, 0xc4213f2690) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 174 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4225e2210) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 175 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4225e2210) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 108 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420132300, 0xc422ac3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).fill(0xc421602e40) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).ReadByte(0xc421602e40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422260b70, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422e47240) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421602f00, 0x16e4b60, 0xc422e472a0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421602f00, 0x16e4b60, 0xc422e472a0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421dc7c40, 0x16e4b60, 0xc422e472a0, 0xc422e472a0, 0xc4205b4ac8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421dc7c40, 0xc422e472a0, 0x2d, 0xc4224d6b40) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequestHeader(0xc4205b4aa0, 0x1d72340, 0xc421dc7c40, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a9c20, 0x8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequest(0xc4205b4aa0, 0x1d72340, 0xc421dc7c40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc4205b4aa0, 0x1d72340, 0xc421dc7c40, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc420132300, 0xc4205b4aa0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 109 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421d34000, 0xc422980d10, 0x30, 0xf9866a) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421d34000, 0x1c61988, 0xc42030cf00, 0x1d71840, 0xc420fe2200) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc420202028, 0xc422d83950) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc420202028, 0xc422d83950) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 195 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4225e22c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 196 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4225e22c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 197 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421400b40, 0xc421a16000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).fill(0xc420b07020) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).ReadByte(0xc420b07020, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42221f2c0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4219100e0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420b07080, 0x16e4b60, 0xc421910140) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420b07080, 0x16e4b60, 0xc421910140, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422e49500, 0x16e4b60, 0xc421910140, 0xc421910140, 0xc422d83108) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422e49500, 0xc421910140, 0x2d, 0xc422e55b00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequestHeader(0xc422d830e0, 0x1d72340, 0xc422e49500, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9c20, 0x8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequest(0xc422d830e0, 0x1d72340, 0xc422e49500, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc422d830e0, 0x1d72340, 0xc422e49500, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc421400b40, 0xc422d830e0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 129 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc420022000, 0xc4201320c0, 0x185fd3, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:171 +0x7a2 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc420022000, 0xc4201320c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:135 +0x258 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420022000, 0xc4203af520) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 202 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc420022000, 0xc4201320c0, 0xc422c0a8a0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420022000, 0xc4219107e0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 211 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42211a160, 0xc420dcfd10, 0x30, 0xf9866a) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42211a160, 0x1c61988, 0xc42030cf00, 0x1d71840, 0xc420fe2200) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc42020dd00, 0xc4205b4870) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc42020dd00, 0xc4205b4870) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 204 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc420022000, 0xc420132240, 0xc422c0aa20) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420022000, 0xc421910860) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 214 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42211a160) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 248 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc422936000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 216 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42211a210, 0xc420ee9d10, 0x30, 0xf9866a) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42211a210, 0x1c61988, 0xc42030cf00, 0x1d71840, 0xc420fe2200) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc4203e40e0, 0xc4205b4a00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d320, 0xc4203e40e0, 0xc4205b4a00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 217 [IO wait]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.runtime_pollWait(0x7f07c646b540, 0x72, 0xc421241ce8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).wait(0xc42271e198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*pollDesc).waitRead(0xc42271e198, 0xc4226d7000, 0x1000, 0x1000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: internal/poll.(*FD).Read(0xc42271e180, 0xc4226d7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*netFD).Read(0xc42271e180, 0xc4226d7000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net.(*conn).Read(0xc4203e40e0, 0xc4226d7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).Read(0xc4203836e0, 0xc42242dc60, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadAtLeast(0x1d59aa0, 0xc4203836e0, 0xc42242dc60, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: io.ReadFull(0x1d59aa0, 0xc4203836e0, 0xc42242dc60, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42211a210, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42211a210) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 218 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42211a210) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 219 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42211a210) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 220 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421fb89c0, 0xc42295b700, 0x20, 0x20, 0x169d4e0, 0xc4205c4780) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4222b5920, 0x1d718c0, 0xc421fb89c0, 0x0, 0xc421fb8900) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42030cf00, 0xc42295b950, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421fb8000, 0xc421c8ce80, 0xc4222b57d0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: reflect.Value.call(0xc420591c80, 0xc4203e4408, 0x13, 0x1bda9a0, 0x4, 0xc420dc8d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: reflect.Value.Call(0xc420591c80, 0xc4203e4408, 0x13, 0xc420dc8d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*service).call(0xc42273ae00, 0xc4205b4aa0, 0xc422e44fa8, 0x0, 0xc421c8ca80, 0xc4203afe40, 0x19758e0, 0xc421c8ce80, 0x16, 0x169e020, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc4205b4aa0, 0x1d72340, 0xc42273ae40, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc421400fc0, 0xc4205b4aa0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 249 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc420fce3c0, 0x7f07c6410ec0, 0xc422500000, 0x1, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:248 +0x2c1 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc420fce3c0, 0x7f07c6410ec0, 0xc422500000, 0x7f07c6410ec0, 0xc422500000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsImpl(0xc422936000, 0xc4224c1e60, 0xc421a0b180, 0x438e9d93001a9b45, 0xc4229aee28, 0x438e9d93001a9b45, 0x1afe340, 0xc4224c1e70) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:756 +0x89 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).(github.com/hashicorp/nomad/nomad/deploymentwatcher.getAllocsImpl)-fm(0xc4224c1e60, 0xc421a0b180, 0xc4203b5e60, 0xc422ccc098, 0xc4224c1a00, 0xc421e97f60, 0xc4203b5e60) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x3e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203b65a0, 0xc422986f20, 0x1, 0x1d71840, 0xc422500000, 0xf28cc9, 0xc4209a9f80, 0x1d71840, 0xc42253f780, 0x1) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:149 +0x114 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc422936000, 0x1, 0xf5, 0xc4216f6200, 0xf6, 0xc4229e0140, 0xf7, 0xc422b79200) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc422936000, 0x1, 0xc42261cd80) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 231 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4204e1680) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 207 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421401200, 0xc422a41000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).fill(0xc421e66d20) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).ReadByte(0xc421e66d20, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42229aab0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421910d80) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421e66d80, 0x16e4b60, 0xc421910de0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421e66d80, 0x16e4b60, 0xc421910de0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4223d2280, 0x16e4b60, 0xc421910de0, 0xc421910de0, 0xc422d83a18) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4223d2280, 0xc421910de0, 0x2d, 0xc422e55000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequestHeader(0xc422d839f0, 0x1d72340, 0xc4223d2280, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9c20, 0x8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequest(0xc422d839f0, 0x1d72340, 0xc4223d2280, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc422d839f0, 0x1d72340, 0xc4223d2280, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc421401200, 0xc422d839f0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 230 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420ca8ae0, 0xc4229bd700, 0x20, 0x20, 0x169d4e0, 0xc4205c4780) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42248b2c0, 0x1d718c0, 0xc420ca8ae0, 0x0, 0xc420ca8a00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42030cf00, 0xc4229bd950, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4219fb620, 0xc421dbb300, 0xc42248b1a0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: reflect.Value.call(0xc420591c80, 0xc42000fcc8, 0x13, 0x1bda9a0, 0x4, 0xc420013d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: reflect.Value.Call(0xc420591c80, 0xc42000fcc8, 0x13, 0xc420013d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*service).call(0xc421dc7000, 0xc422d830e0, 0xc4226009a8, 0x0, 0xc4227d9300, 0xc422e47f20, 0x19758e0, 0xc421dbb300, 0x16, 0x169e020, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc422d830e0, 0x1d72340, 0xc42253f140, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc4204cf5c0, 0xc422d830e0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 260 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421401440, 0xc422c5f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).fill(0xc421e67f80) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: bufio.(*Reader).ReadByte(0xc421e67f80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4224d0960, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4219117a0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42100c000, 0x16e4b60, 0xc421911800) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42100c000, 0x16e4b60, 0xc421911800, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4223d2d00, 0x16e4b60, 0xc421911800, 0xc421911800, 0xc4205b4938) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4223d2d00, 0xc421911800, 0x2d, 0xc422e55960) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequestHeader(0xc4205b4910, 0x1d72340, 0xc4223d2d00, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9c20, 0x8) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).readRequest(0xc4205b4910, 0x1d72340, 0xc4223d2d00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: net/rpc.(*Server).ServeRequest(0xc4205b4910, 0x1d72340, 0xc4223d2d00, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42030cf00, 0x1d71840, 0xc420fe2200, 0x1d7d0e0, 0xc421401440, 0xc4205b4910) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 261 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4223d3440) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 262 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc420022000, 0xc420132240, 0x1d6eb80, 0xc4223d3440, 0xc422c0b6e0, 0xc422c0b740) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420022000, 0xc4223d3480) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 232 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4204e1680) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 233 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4200a9ef0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc42030cf00) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 234 [semacquire]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: sync.runtime_SemacquireMutex(0xc4204e062c, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: sync.(*Mutex).Lock(0xc4204e0628) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: sync.(*RWMutex).Lock(0xc4204e0628) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4204e05a0, 0x1d71840, 0xc42253f740) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 235 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420250990) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 236 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420250990) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 237 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc42248b8c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 238 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42253f780, 0xc422985ce8, 0x20, 0x20, 0x18, 0xc421a0b120) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224c1d10, 0x1d71840, 0xc42253f780, 0xc421a0b120, 0x184987) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203b65a0, 0xc4229b9ed0, 0x184987, 0x1d71840, 0xc42253f780, 0xc4209a9fe0, 0xc42248b890, 0xc420be5530, 0x24, 0x2bba400) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc4209a9f80, 0x1d71840, 0xc42253f780, 0x184987, 0xc422215f00, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc4209a9f80, 0x1d71840, 0xc42253f780) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 239 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42253f800, 0xc42299d770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224c1e90, 0x1d71840, 0xc42253f800, 0x0, 0x1849b5) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203b65a0, 0xc42299d960, 0x1849b5, 0x1d71840, 0xc42253f800, 0x552bde, 0xc42299d950, 0xc42299d970, 0xf33ba5, 0xc420416918) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc4204168c0, 0x1d71840, 0xc42253f800, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc4204168c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 240 [runnable]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodesImpl(0xc42253f880, 0xc4224d1b00, 0xc421911cc0, 0x877a8bb80e220241, 0xc420598b78, 0x877a8bb80e220241, 0x1afe340, 0xc4224d1b10) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:247 +0x12f Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).(github.com/hashicorp/nomad/nomad/drainer.getNodesImpl)-fm(0xc4224d1b00, 0xc421911cc0, 0xc4203b5e60, 0xc422c32a48, 0x0, 0x0, 0x100010000) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0x3e Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203b65a0, 0xc420598c68, 0x1833a4, 0x1d71840, 0xc42253f7c0, 0x552bde, 0xc4200a9cc0, 0x2, 0xc420486d40, 0x40) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:149 +0x114 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc42253f880, 0x1833a4, 0x3b, 0xc420598de8, 0x1, 0x1) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc42253f880) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 241 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc42253f8c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 274 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420a0a5a0, 0x1d71840, 0xc42253f7c0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 263 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4224d18c0, 0xc422c0b7a0) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: goroutine 264 [select]: Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4224d18c0, 0xc422c0b800) Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:14 ip-172-24-20-236 nomad.sh[25589]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:14 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:27:14 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:27:14 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: ==> Starting Nomad agent... Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: ==> Nomad agent configuration: Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: Client: false Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: Log Level: INFO Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: Region: amrv01 (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: Server: true Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: Version: 0.8.4 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.223713 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.225654 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: Attempting re-join to previously known node: consulnomad-02.aav01: 172.27.0.42:4648 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580002 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580027 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580039 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580050 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580062 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580075 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580343 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580360 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580372 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580383 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580395 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580406 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [WARN] memberlist: Refuting an alive message Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15 [INFO] serf: Re-joined to previously known node: consulnomad-02.aav01: 172.27.0.42:4648 Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580974 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580987 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.580998 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.581008 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.581019 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.581029 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:27:15 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:15.581040 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:17 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:17 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:18 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:18 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:27:18 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:18 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 29 Aug 23 15:27:18 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:18 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19.037947 [INFO] nomad: cluster leadership acquired Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 501.46679ms Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:19 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:19 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:20 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:20 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 992.102751ms Aug 23 15:27:20 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:20 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:20 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:20 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:20 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:20 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 1.492082377s Aug 23 15:27:20 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:20 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:20 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:20 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:21 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:21 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:21 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:21.681169 [INFO] server.nomad: successfully contacted 19 Nomad Servers Aug 23 15:27:21 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:21 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:23 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:23 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:24 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:24 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:25 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:25 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:26.051221 [ERR] worker: failed to update evaluation : evaluation token does not match Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: 2018/08/23 15:27:26.051273 [ERR] worker: failed to nack evaluation '77c1aa2d-9be0-536f-116a-0e8c62a0fefa': Token does not match for Evaluation ID Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: panic: runtime error: invalid memory address or nil pointer dereference Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xf55789] Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: goroutine 158 [running]: Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Ack(0xc4200c6140, 0xc42089b5f0, 0x24, 0xc422cc8810, 0x24, 0x0, 0x0) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:552 +0x2b9 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: github.com/hashicorp/nomad/nomad.(*Eval).Ack(0xc4203240c8, 0xc422217200, 0xc4225e93b8, 0x0, 0x0) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:163 +0x1c1 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: reflect.Value.call(0xc420280cc0, 0xc420324370, 0x13, 0x1bda9a0, 0x4, 0xc422c5dca8, 0x3, 0x3, 0x18deac0, 0x19, ...) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: reflect.Value.Call(0xc420280cc0, 0xc420324370, 0x13, 0xc422c5dca8, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: net/rpc.(*service).call(0xc420083a40, 0xc4200a90e0, 0xc4225e93b0, 0x0, 0xc420330980, 0xc4207ae1a0, 0x1974560, 0xc422217200, 0x16, 0x169db60, ...) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: net/rpc.(*Server).ServeRequest(0xc4200a90e0, 0x1d71b80, 0xc421a4ec40, 0x18deac0, 0x1) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420568000, 0x1be0806, 0x8, 0x1974560, 0xc4222171a0, 0x169db60, 0xc4225e93a8, 0x2b96de0, 0x76bff1) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: github.com/hashicorp/nomad/nomad.(*Worker).sendAck(0xc4214f73e0, 0xc42089b5f0, 0x24, 0xc422cc8810, 0x24, 0x1) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:216 +0x274 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214f73e0) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:133 +0x1b3 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25609]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:26 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:27:26 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:27:26 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:27:26 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:27:26 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:27:26 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: ==> Starting Nomad agent... Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: ==> Nomad agent configuration: Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: Client: false Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: Log Level: INFO Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: Region: amrv01 (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: Server: true Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: Version: 0.8.4 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.472994 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: Attempting re-join to previously known node: ip-172-29-20-70.atf01: 172.29.20.70:4648 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.473129 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [WARN] memberlist: Refuting an alive message Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664004 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664029 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664042 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664053 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664063 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664073 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664192 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664206 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664247 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 2018/08/23 15:27:26.664257 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664356 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664371 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664765 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664782 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664793 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664803 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664813 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664823 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26.664833 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:26 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:26 [INFO] serf: Re-joined to previously known node: ip-172-29-20-70.atf01: 172.29.20.70:4648 Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 30 Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27.780371 [INFO] nomad: cluster leadership acquired Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:27 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:27 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: 2018/08/23 15:27:28 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 1.193388112s Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: fatal error: concurrent map writes Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 244 [running]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc42129dd08 sp=0xc42129dce8 pc=0x42d161 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: runtime.mapdelete(0x1844be0, 0xc420015f50, 0xc42129de20) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc42129dd68 sp=0xc42129dd08 pc=0x40a761 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc420019ac0, 0x1d6dd40, 0xc42000e820, 0x0, 0xed3109f30) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc42129de50 sp=0xc42129dd68 pc=0xf2164c Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4204f9360, 0x1d71840, 0xc4226e3bc0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc42129dfc8 sp=0xc42129de50 pc=0xf56bb7 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: runtime.goexit() Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc42129dfd0 sp=0xc42129dfc8 pc=0x45c711 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 1 [select]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420358fc0, 0xc4200ee3c0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420358fc0, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc42047dcc0, 0xc42047dcc0, 0xc420424fc0, 0x37) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204e3f68) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: main.main() Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 5 [syscall]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: os/signal.signal_recv(0x0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: os/signal.loop() Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: created by os/signal.init.0 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 139 [select]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4203ec000, 0xc420426cc0, 0xc420c70a80) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:351 +0x4fe Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203ec000, 0xc420437d40) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 193 [select]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42116c000, 0xc4212a0d10, 0x30, 0xf9866a) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42116c000, 0x1c61988, 0xc4204a0f00, 0x1d71840, 0xc420c7ae80) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d320, 0xc42000e030, 0xc4205f8140) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d320, 0xc42000e030, 0xc4205f8140) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 38 [select]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc420082700, 0x1d71840, 0xc4204f50c0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 31 [select, locked to thread]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: runtime.selectgo(0xc420055f50, 0xc4200906c0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: runtime.ensureSigM.func1() Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: runtime.goexit() Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 32 [select]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc420082280) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 33 [sleep]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: time.Sleep(0x3b9aca00) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc42044b550) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 66 [select]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420426600) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 67 [select]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4200873e0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: goroutine 204 [select]: Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4210c4180, 0xc421473000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).fill(0xc4210d98c0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).ReadByte(0xc4210d98c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42182b980, 0x0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4204d23c0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210d9920, 0x16e4b60, 0xc4204d2440) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210d9920, 0x16e4b60, 0xc4204d2440, 0x0, 0x0) Aug 23 15:27:28 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421e40c80, 0x16e4b60, 0xc4204d2440, 0xc4204d2440, 0xc421d50898) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421e40c80, 0xc4204d2440, 0x2d, 0xc4217eb620) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequestHeader(0xc421d50870, 0x1d72340, 0xc421e40c80, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9a40, 0x8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequest(0xc421d50870, 0x1d72340, 0xc421e40c80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).ServeRequest(0xc421d50870, 0x1d72340, 0xc421e40c80, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d0e0, 0xc4210c4180, 0xc421d50870) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 69 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4203d48f0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4205263c0, 0xc4205220e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 35 [IO wait]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.runtime_pollWait(0x7f3144536f00, 0x72, 0xc4205659a8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).wait(0xc42046c798, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).waitRead(0xc42046c798, 0xc4203ee000, 0x1000, 0x1000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*FD).Read(0xc42046c780, 0xc4203ee000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*netFD).Read(0xc42046c780, 0xc4203ee000, 0x1000, 0x1000, 0x458600, 0xc4202ef500, 0x4) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*conn).Read(0xc420204020, 0xc4203ee000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/http.(*persistConn).Read(0xc4200b27e0, 0xc4203ee000, 0x1000, 0x1000, 0xc420565b98, 0x405675, 0xc420090240) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).fill(0xc420086180) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).Peek(0xc420086180, 0x1, 0x0, 0x0, 0x1, 0xc420e51200, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/http.(*persistConn).readLoop(0xc4200b27e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by net/http.(*Transport).dialConn Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 201 [IO wait]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.runtime_pollWait(0x7f31445366e0, 0x72, 0xc42121ace8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).wait(0xc420a4c398, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).waitRead(0xc420a4c398, 0xc421472000, 0x1000, 0x1000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*FD).Read(0xc420a4c380, 0xc421472000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*netFD).Read(0xc420a4c380, 0xc421472000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*conn).Read(0xc420216210, 0xc421472000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).Read(0xc42125bc80, 0xc42067d680, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: io.ReadAtLeast(0x1d59aa0, 0xc42125bc80, 0xc42067d680, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: io.ReadFull(0x1d59aa0, 0xc42125bc80, 0xc42067d680, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420ec2000, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420ec2000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 138 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204263c0, 0xc421052000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).fill(0xc420aa7620) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).ReadByte(0xc420aa7620, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4210482d0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420436ba0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420aa7680, 0x16e4b60, 0xc420436c00) Aug 23 15:27:29 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420aa7680, 0x16e4b60, 0xc420436c00, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420aa47c0, 0x16e4b60, 0xc420436c00, 0xc420436c00, 0xc420bf6348) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420aa47c0, 0xc420436c00, 0x2d, 0xc422ed0b60) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequestHeader(0xc420bf6320, 0x1d72340, 0xc420aa47c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9a40, 0x8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequest(0xc420bf6320, 0x1d72340, 0xc420aa47c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).ServeRequest(0xc420bf6320, 0x1d72340, 0xc420aa47c0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d0e0, 0xc4204263c0, 0xc420bf6320) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 203 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420ec2000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 36 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/http.(*persistConn).writeLoop(0xc4200b27e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by net/http.(*Transport).dialConn Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 70 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4203a4000, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc420238160) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 87 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4203ec000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4203ec000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4203ec000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203ec000, 0xc4205104e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 243 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4200a9d10, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4204a0f00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 88 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4203ec000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203ec000, 0xc4205104f0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 202 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420ec2000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 200 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420ec2000, 0xc42114fd10, 0x30, 0xf9866a) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420ec2000, 0x1c61988, 0xc4204a0f00, 0x1d71840, 0xc420c7ae80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d320, 0xc420216210, 0xc421d507d0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d320, 0xc420216210, 0xc421d507d0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 171 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4203ec000, 0xc420426d80, 0xc420aeb3e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203ec000, 0xc420425660) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 89 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4203ec000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203ec000, 0xc420510500) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 90 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420c35e30) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 91 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4203c05b0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 92 [IO wait]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.runtime_pollWait(0x7f3144536d60, 0x72, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).wait(0xc420ebb598, 0x72, 0xc4203c2000, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).waitRead(0xc420ebb598, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*FD).Accept(0xc420ebb580, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*netFD).accept(0xc420ebb580, 0xc420216208, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*TCPListener).accept(0xc4204ee4b8, 0x458e40, 0xc420051f90, 0xc420051f98) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*TCPListener).AcceptTCP(0xc4204ee4b8, 0x1c63658, 0xc420143860, 0xc420216208) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420143860) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 93 [IO wait]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.runtime_pollWait(0x7f3144536c90, 0x72, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).wait(0xc420ebb618, 0x72, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).waitRead(0xc420ebb618, 0xc42280a000, 0x10000, 0x10000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*FD).ReadFrom(0xc420ebb600, 0xc42280a000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*netFD).readFrom(0xc420ebb600, 0xc42280a000, 0x10000, 0x10000, 0xeb, 0xeb, 0x1d62b40, 0x412539, 0xc421154e40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*UDPConn).readFrom(0xc4204ee4d8, 0xc42280a000, 0x10000, 0x10000, 0x101, 0xc421154e18, 0xc421154e90, 0x443cf7) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*UDPConn).ReadFrom(0xc4204ee4d8, 0xc42280a000, 0x10000, 0x10000, 0x10000, 0x10000, 0x6a36, 0x94c38290, 0x2b96de0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420143860) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 94 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420143860) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 95 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420143860, 0x12a05f200, 0xc4214e6d20, 0xc420f5a960, 0xc420510ab0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 96 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420143860, 0xc420f5a960) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 97 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420143860, 0x1dcd6500, 0xc4214e6d80, 0xc420f5a960, 0xc420510ac0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 146 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4202fcf00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 147 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4202fcf00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 148 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202fcf00, 0x1bdd09a, 0x6, 0xc420c35f80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 149 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202fcf00, 0x1bdba6c, 0x5, 0xc420c35fb0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 150 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202fcf00, 0x1bdbc6a, 0x5, 0xc420b2a000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 145 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4204f90e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 152 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4214e6de0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214e6de0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214e6de0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 153 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4214e6e40, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214e6e40, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214e6e40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 154 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4214e6ea0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214e6ea0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214e6ea0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 155 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4214e6f00, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214e6f00, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214e6f00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 156 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420e80000, 0xc4204a0f00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 157 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4204a0f00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 158 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4204a0f00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 159 [IO wait]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.runtime_pollWait(0x7f3144536e30, 0x72, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).wait(0xc42056ed18, 0x72, 0xc42020c600, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).waitRead(0xc42056ed18, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*FD).Accept(0xc42056ed00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*netFD).accept(0xc42056ed00, 0x1, 0x1, 0xc42056af20) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*TCPListener).accept(0xc4204eec08, 0x406b4a, 0xc420ea85a0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*TCPListener).Accept(0xc4204eec08, 0xc420ea85a0, 0x0, 0x3f800000, 0xc420c7ae80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4204a0f00, 0x1d71840, 0xc420c7ae80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 160 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4204f9360, 0x3b9aca00, 0xc420090b40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 161 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4200a9d10, 0x3b9aca00, 0xc420090b40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 162 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc42044cbd0, 0x3b9aca00, 0xc420090b40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 163 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4203d48f0, 0x3b9aca00, 0xc420090b40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 164 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4204a0f00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 165 [IO wait]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.runtime_pollWait(0x7f3144536bc0, 0x72, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).wait(0xc420ebbc98, 0x72, 0xc4204f5100, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).waitRead(0xc420ebbc98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*FD).Accept(0xc420ebbc80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*netFD).accept(0xc420ebbc80, 0x402037, 0xc420229de0, 0x1805ec0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*TCPListener).accept(0xc4204ef708, 0x458e40, 0xc421152e90, 0xc421152e98) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*TCPListener).Accept(0xc4204ef708, 0x1c65af8, 0xc420229d60, 0x1d71900, 0xc420a421b0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/http.(*Server).Serve(0xc42049c000, 0x1d6fc00, 0xc4204ef708, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/http.Serve(0x1d6fc00, 0xc4204ef708, 0x1d5df00, 0xc420a420f0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420c7af40, 0x1d6fc00, 0xc4204ef708, 0xc420511100, 0xc420e800c0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 166 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420ebbd00, 0xc420511110, 0xc420f5ad80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 167 [sleep]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: time.Sleep(0x2f0a2a596) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc420358fc0, 0xc420ebbd00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 212 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4228f3e30, 0xc4226ca8a0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 116 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420238210, 0xc421146d10, 0x30, 0xf9866a) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420238210, 0x1c61988, 0xc4204a0f00, 0x1d71840, 0xc420c7ae80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d320, 0xc420204058, 0xc420bf6230) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d320, 0xc420204058, 0xc420bf6230) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 117 [IO wait]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.runtime_pollWait(0x7f3144536a20, 0x72, 0xc42114ece8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).wait(0xc420cb4198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).waitRead(0xc420cb4198, 0xc421170000, 0x1000, 0x1000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*FD).Read(0xc420cb4180, 0xc421170000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*netFD).Read(0xc420cb4180, 0xc421170000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*conn).Read(0xc420204058, 0xc421170000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).Read(0xc420bf2ba0, 0xc42070a8a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: io.ReadAtLeast(0x1d59aa0, 0xc420bf2ba0, 0xc42070a8a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: io.ReadFull(0x1d59aa0, 0xc420bf2ba0, 0xc42070a8a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420238210, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420238210) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 118 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420238210) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 119 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420238210) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 185 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4210c40c0, 0xc42067f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).fill(0xc420573260) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).ReadByte(0xc420573260, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420e41770, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4201a7b80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4205732c0, 0x16e4b60, 0xc4201a7be0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4205732c0, 0x16e4b60, 0xc4201a7be0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4208bb3c0, 0x16e4b60, 0xc4201a7be0, 0xc4201a7be0, 0xc420bf6348) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4208bb3c0, 0xc4201a7be0, 0x2d, 0xc422ed0bc0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequestHeader(0xc420bf6320, 0x1d72340, 0xc4208bb3c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9a40, 0x8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequest(0xc420bf6320, 0x1d72340, 0xc4208bb3c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).ServeRequest(0xc420bf6320, 0x1d72340, 0xc4208bb3c0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d0e0, 0xc4210c40c0, 0xc420bf6320) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 211 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4228f3e30, 0xc4226ca840) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 199 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421730180, 0xc421996000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).fill(0xc4219ac300) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).ReadByte(0xc4219ac300, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421786780, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420513e60) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4219ac360, 0x16e4b60, 0xc420513f00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4219ac360, 0x16e4b60, 0xc420513f00, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4222405c0, 0x16e4b60, 0xc420513f00, 0xc420513f00, 0xc420bf6348) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4222405c0, 0xc420513f00, 0x2d, 0xc4217eade0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequestHeader(0xc420bf6320, 0x1d72340, 0xc4222405c0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200a9a40, 0x8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequest(0xc420bf6320, 0x1d72340, 0xc4222405c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).ServeRequest(0xc420bf6320, 0x1d72340, 0xc4222405c0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d0e0, 0xc421730180, 0xc420bf6320) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 242 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4204f90e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 126 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420238840, 0xc421148d10, 0x30, 0xf9866a) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420238840, 0x1c61988, 0xc4204a0f00, 0x1d71840, 0xc420c7ae80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d320, 0xc420204530, 0xc420bf6d20) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d320, 0xc420204530, 0xc420bf6d20) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 127 [IO wait]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.runtime_pollWait(0x7f3144536950, 0x72, 0xc421155ce8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).wait(0xc420a90518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).waitRead(0xc420a90518, 0xc42104f000, 0x1000, 0x1000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*FD).Read(0xc420a90500, 0xc42104f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*netFD).Read(0xc420a90500, 0xc42104f000, 0x1000, 0x1000, 0xc4200555d8, 0x1, 0xc420055608) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*conn).Read(0xc420204530, 0xc42104f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).Read(0xc4210b82a0, 0xc420798dc0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: io.ReadAtLeast(0x1d59aa0, 0xc4210b82a0, 0xc420798dc0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: io.ReadFull(0x1d59aa0, 0xc4210b82a0, 0xc420798dc0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420238840, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420238840) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 128 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420238840) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 129 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420238840) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 178 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4210c4000, 0xc4210d0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).fill(0xc4210ce6c0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).ReadByte(0xc4210ce6c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4210ca810, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42037d080) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210ce720, 0x16e4b60, 0xc42037d200) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210ce720, 0x16e4b60, 0xc42037d200, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e31480, 0x16e4b60, 0xc42037d200, 0xc42037d200, 0xc420bf6de8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e31480, 0xc42037d200, 0x2d, 0xc420880440) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequestHeader(0xc420bf6dc0, 0x1d72340, 0xc420e31480, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200a9a40, 0x8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequest(0xc420bf6dc0, 0x1d72340, 0xc420e31480, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).ServeRequest(0xc420bf6dc0, 0x1d72340, 0xc420e31480, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d0e0, 0xc4210c4000, 0xc420bf6dc0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 183 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4203ec000, 0xc420426d80, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4203ec000, 0xc420426d80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203ec000, 0xc42055ca20) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 182 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4203ec000, 0xc420426cc0, 0x186076, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:171 +0x7a2 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4203ec000, 0xc420426cc0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:135 +0x258 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203ec000, 0xc42055c9e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 170 [runnable]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldEnqueue(0xc4215186e0, 0x1a1c020) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6594 +0x2a0 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4204a0f00, 0xc420437201, 0xc4204372e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:298 +0x117 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4204a0f00, 0xc420aeb200, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4204a0f00, 0xc420aeb200) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc42072c070, 0xc4204a0f00, 0xc420aeb200) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 172 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420e31cc0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 173 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4203ec000, 0xc420426d80, 0x1d6eb80, 0xc420e31cc0, 0xc420aebb60, 0xc420aebc20) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203ec000, 0xc420e31d40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 210 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4227e5ec0, 0xc42117b700, 0x20, 0x20, 0x169d4e0, 0xc42055d200) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4228f3740, 0x1d718c0, 0xc4227e5ec0, 0x0, 0xc4227e5e00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4204a0f00, 0xc42117b950, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4210d91a0, 0xc420778f80, 0xc4228f3620, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: reflect.Value.call(0xc42056c7e0, 0xc420216708, 0x13, 0x1bda9a0, 0x4, 0xc420062d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: reflect.Value.Call(0xc42056c7e0, 0xc420216708, 0x13, 0xc420062d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*service).call(0xc421e40c00, 0xc421d50870, 0xc422ed5938, 0x0, 0xc4223e3700, 0xc422bf1160, 0x19758e0, 0xc420778f80, 0x16, 0x169e020, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).ServeRequest(0xc421d50870, 0x1d72340, 0xc422b2cc40, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d0e0, 0xc420426b40, 0xc421d50870) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 186 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200c02c0, 0xc42114dd10, 0x30, 0xf9866a) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200c02c0, 0x1c61988, 0xc4204a0f00, 0x1d71840, 0xc420c7ae80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d320, 0xc420204508, 0xc4207b05a0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d320, 0xc420204508, 0xc4207b05a0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 187 [IO wait]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.runtime_pollWait(0x7f3144536880, 0x72, 0xc42114bce8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).wait(0xc4212fc418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).waitRead(0xc4212fc418, 0xc42124e000, 0x1000, 0x1000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*FD).Read(0xc4212fc400, 0xc42124e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*netFD).Read(0xc4212fc400, 0xc42124e000, 0x1000, 0x1000, 0xc42156add8, 0x1, 0xc42156ae08) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*conn).Read(0xc420204508, 0xc42124e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).Read(0xc420573560, 0xc4201720e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: io.ReadAtLeast(0x1d59aa0, 0xc420573560, 0xc4201720e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: io.ReadFull(0x1d59aa0, 0xc420573560, 0xc4201720e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200c02c0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200c02c0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 188 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200c02c0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 189 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200c02c0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 190 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4210b9aa0, 0xc42126b700, 0x20, 0x20, 0x169d4e0, 0xc42055d200) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4212e9cb0, 0x1d718c0, 0xc4210b9aa0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4204a0f00, 0xc42126b950, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4210b9440, 0xc4215e3100, 0xc420fed590, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: reflect.Value.call(0xc42056c7e0, 0xc420204df8, 0x13, 0x1bda9a0, 0x4, 0xc42126bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: reflect.Value.Call(0xc42056c7e0, 0xc420204df8, 0x13, 0xc42126bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*service).call(0xc4208bbf00, 0xc4207b0640, 0xc42086a3a0, 0x0, 0xc4215e2d00, 0xc4201a7dc0, 0x19758e0, 0xc4215e3100, 0x16, 0x169e020, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).ServeRequest(0xc4207b0640, 0x1d72340, 0xc4208bbf40, 0x0, 0xc420de2300) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d0e0, 0xc420de2300, 0xc4207b0640) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 197 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421730480, 0xc42266e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).fill(0xc4202ff320) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).ReadByte(0xc4202ff320, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d965a0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4201a93c0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4202ff380, 0x16e4b60, 0xc4201a9420) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4202ff380, 0x16e4b60, 0xc4201a9420, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421d43a40, 0x16e4b60, 0xc4201a9420, 0xc4201a9420, 0xc420bf6348) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421d43a40, 0xc4201a9420, 0x2d, 0xc422ed0c40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequestHeader(0xc420bf6320, 0x1d72340, 0xc421d43a40, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a9a40, 0x8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequest(0xc420bf6320, 0x1d72340, 0xc421d43a40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).ServeRequest(0xc420bf6320, 0x1d72340, 0xc421d43a40, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d0e0, 0xc421730480, 0xc420bf6320) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 226 [IO wait]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.runtime_pollWait(0x7f3144536610, 0x72, 0xc42114cce8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).wait(0xc4223ba018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*pollDesc).waitRead(0xc4223ba018, 0xc421026000, 0x1000, 0x1000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: internal/poll.(*FD).Read(0xc4223ba000, 0xc421026000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*netFD).Read(0xc4223ba000, 0xc421026000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net.(*conn).Read(0xc42000e030, 0xc421026000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).Read(0xc42056d020, 0xc422ed5940, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: io.ReadAtLeast(0x1d59aa0, 0xc42056d020, 0xc422ed5940, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: io.ReadFull(0x1d59aa0, 0xc42056d020, 0xc422ed5940, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42116c000, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42116c000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 227 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42116c000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 228 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42116c000) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 229 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4210cff80, 0xc422c8f700, 0x20, 0x20, 0x169d4e0, 0xc42055d200) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4213f97a0, 0x1d718c0, 0xc4210cff80, 0x0, 0xc4210cff00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4204a0f00, 0xc422c8f950, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4210cf440, 0xc422671880, 0xc4213f95c0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: reflect.Value.call(0xc42056c7e0, 0xc42000e7b0, 0x13, 0x1bda9a0, 0x4, 0xc42121cd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: reflect.Value.Call(0xc42056c7e0, 0xc42000e7b0, 0x13, 0xc42121cd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*service).call(0xc421d432c0, 0xc4205f81e0, 0xc42067dff8, 0x0, 0xc422671480, 0xc4204369c0, 0x19758e0, 0xc422671880, 0x16, 0x169e020, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).ServeRequest(0xc4205f81e0, 0x1d72340, 0xc421d434c0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d0e0, 0xc420426fc0, 0xc4205f81e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 230 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420de3140, 0xc420f91000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).fill(0xc4210cfec0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: bufio.(*Reader).ReadByte(0xc4210cfec0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4213f93b0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420436c80) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210cff20, 0x16e4b60, 0xc420436d40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210cff20, 0x16e4b60, 0xc420436d40, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421d43680, 0x16e4b60, 0xc420436d40, 0xc420436d40, 0xc4205f8208) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421d43680, 0xc420436d40, 0x2d, 0xc422ed0fc0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequestHeader(0xc4205f81e0, 0x1d72340, 0xc421d43680, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a9a40, 0x8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).readRequest(0xc4205f81e0, 0x1d72340, 0xc421d43680, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: net/rpc.(*Server).ServeRequest(0xc4205f81e0, 0x1d72340, 0xc421d43680, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0f00, 0x1d71840, 0xc420c7ae80, 0x1d7d0e0, 0xc420de3140, 0xc4205f81e0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 245 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc42044cbd0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 246 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc42044cbd0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 247 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc42234c330) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 248 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4226e3c00, 0xc422b9ece8, 0x20, 0x20, 0x18, 0xc422bf16a0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422362030, 0x1d71840, 0xc4226e3c00, 0xc422bf16a0, 0x184987) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204372e0, 0xc422c8bed0, 0x184987, 0x1d71840, 0xc4226e3c00, 0xc420ebb7e0, 0xc42234c300, 0xc420f72750, 0x24, 0x2bba400) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420ebb780, 0x1d71840, 0xc4226e3c00, 0x184987, 0xc422b2d240, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420ebb780, 0x1d71840, 0xc4226e3c00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 249 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4226e3c80, 0xc4214f1770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422362450, 0x1d71840, 0xc4226e3c80, 0x0, 0x1849b5) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204372e0, 0xc4214f1960, 0x1849b5, 0x1d71840, 0xc4226e3c80, 0x552bde, 0xc4214f1950, 0xc4214f1970, 0xf33ba5, 0xc4203762f8) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc4203762a0, 0x1d71840, 0xc4226e3c80, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc4203762a0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 250 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4226e3c40, 0xc422b7ba78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42234d1d0, 0x1d71840, 0xc4226e3c40, 0xc42234d3b0, 0x1833a4) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204372e0, 0xc422b7bc68, 0x1833a4, 0x1d71840, 0xc4226e3c40, 0x552bde, 0xc4200a9ae0, 0x2, 0xc420353580, 0x40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc4226e3cc0, 0x1833a4, 0x3b, 0xc42129cde8, 0x1, 0x1) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc4226e3cc0) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 251 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc4226e3d00) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 252 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420143a40, 0x1d71840, 0xc4226e3c40) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 213 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc421930140) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: goroutine 215 [select]: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422b2d340, 0xc422b9fd28, 0x20, 0x20, 0xc4201e8800, 0x1bd9a98) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422362420, 0x1d71840, 0xc422b2d340, 0xc420437080, 0x1849b5) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204372e0, 0xc422c91f20, 0x1849b5, 0x1d71840, 0xc422b2d340, 0xf28cc9, 0xc420ebb780, 0x1d71840, 0xc4226e3c00, 0x1) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc421930140, 0x1849b5, 0xd219e1, 0x1, 0x0, 0x0, 0x0, 0xc420f5a960) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc421930140, 0x1849b5, 0xc420bf3980) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25629]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:27:29 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:27:29 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:27:29 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: ==> Starting Nomad agent... Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: ==> Nomad agent configuration: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: Client: false Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: Log Level: INFO Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: Region: amrv01 (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: Server: true Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: Version: 0.8.4 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.468281 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: Attempting re-join to previously known node: ip-172-25-5-210.amv01: 172.25.5.210:4648 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.468413 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.911500 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.911524 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.911538 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.911550 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.911561 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.911573 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.911988 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912009 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912021 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [WARN] memberlist: Refuting an alive message Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912032 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912271 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912283 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912295 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912306 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912318 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29 [INFO] serf: Re-joined to previously known node: ip-172-25-5-210.amv01: 172.25.5.210:4648 Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912407 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912419 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912430 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:29 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:29.912441 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:30 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:30 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 32 Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32.627707 [INFO] nomad: cluster leadership acquired Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:32 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:32 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: 2018/08/23 15:27:33 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: fatal error: concurrent map writes Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 257 [running]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc420065d08 sp=0xc420065ce8 pc=0x42d161 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: runtime.mapdelete(0x1844be0, 0xc420468c90, 0xc420065e20) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/hashmap.go:682 +0x2f6 fp=0xc420065d68 sp=0xc420065d08 pc=0x40a746 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc4202001e0, 0x1d6dd40, 0xc4223ee228, 0x0, 0xed3109f35) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc420065e50 sp=0xc420065d68 pc=0xf2164c Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4204f00a0, 0x1d71840, 0xc422835300) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc420065fc8 sp=0xc420065e50 pc=0xf56bb7 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: runtime.goexit() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420065fd0 sp=0xc420065fc8 pc=0x45c711 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 1 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42031a5b0, 0xc420470640) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42031a5b0, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc420310640, 0xc420310640, 0xc4202bda20, 0x37) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204dbf68) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: main.main() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 5 [syscall]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: os/signal.signal_recv(0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: os/signal.loop() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by os/signal.init.0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 123 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420dc43c0, 0xc420f45000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).fill(0xc420087560) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).ReadByte(0xc420087560, 0x101000000000016, 0x0, 0x2d) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421159770, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421266400) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4200875c0, 0x16e4b60, 0xc421266460) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4200875c0, 0x16e4b60, 0xc421266460, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4211705c0, 0x16e4b60, 0xc421266460, 0xc421266460, 0xc421254118) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4211705c0, 0xc421266460, 0x2d, 0xc420dcd020) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequestHeader(0xc4212540f0, 0x1d72340, 0xc4211705c0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200a8fa0, 0x8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequest(0xc4212540f0, 0x1d72340, 0xc4211705c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).ServeRequest(0xc4212540f0, 0x1d72340, 0xc4211705c0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d0e0, 0xc420dc43c0, 0xc4212540f0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 55 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc42035e3c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 178 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420232210, 0xc420bb7d10, 0x30, 0xf9866a) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420232210, 0x1c61988, 0xc420494780, 0x1d71840, 0xc421464200) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc42020c848, 0xc4210e8960) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc42020c848, 0xc4210e8960) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 54 [sleep]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: time.Sleep(0x3b9aca00) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc4202e2840) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 53 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc4203ac800) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 144 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0e880, 0x72, 0xc4210449a8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420217898, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420217898, 0xc421150000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc420217880, 0xc421150000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc420217880, 0xc421150000, 0x1000, 0x1000, 0x458600, 0xc421263200, 0x4) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc42020c838, 0xc421150000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/http.(*persistConn).Read(0xc4204ceea0, 0xc421150000, 0x1000, 0x1000, 0xc421044b98, 0x405675, 0xc420ef6a20) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).fill(0xc4201f4600) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Peek(0xc4201f4600, 0x1, 0x0, 0x0, 0x1, 0xc4214a40c0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/http.(*persistConn).readLoop(0xc4204ceea0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by net/http.(*Transport).dialConn Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 28 [select, locked to thread]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: runtime.selectgo(0xc420141750, 0xc42038e420) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: runtime.ensureSigM.func1() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: runtime.goexit() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 56 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc42037e360) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 121 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420232210) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 58 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc42021be10) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc420208ae0, 0xc420318330) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 74 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4202f2100, 0x1d71840, 0xc420082040) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 16 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200b4000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200b4000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200b4000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc42051aae0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 11 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0ef00, 0x72, 0xc4200619a8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420216418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420216418, 0xc42046c000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc420216400, 0xc42046c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc420216400, 0xc42046c000, 0x1000, 0x1000, 0x458600, 0xc4202e0180, 0x4) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc42020c018, 0xc42046c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/http.(*persistConn).Read(0xc42042a120, 0xc42046c000, 0x1000, 0x1000, 0xc420061b98, 0x405675, 0xc4202e40c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).fill(0xc4201f4540) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Peek(0xc4201f4540, 0x1, 0x0, 0x0, 0x1, 0xc420ee6780, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/http.(*persistConn).readLoop(0xc42042a120) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by net/http.(*Transport).dialConn Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 145 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/http.(*persistConn).writeLoop(0xc4204ceea0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by net/http.(*Transport).dialConn Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 195 [select]: Aug 23 15:27:33 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421268a80, 0xc420d17700, 0x20, 0x20, 0x169d4e0, 0xc4205333c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420f54570, 0x1d718c0, 0xc421268a80, 0x0, 0xc421268a00) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420494780, 0xc420d17950, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42140c480, 0xc420848000, 0xc420f54420, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: reflect.Value.call(0xc42019ade0, 0xc42000f3a8, 0x13, 0x1bda9a0, 0x4, 0xc420d17d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: reflect.Value.Call(0xc42019ade0, 0xc42000f3a8, 0x13, 0xc420d17d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*service).call(0xc420de0200, 0xc420fa7ae0, 0xc420320e48, 0x0, 0xc420f5f480, 0xc420aee7c0, 0x19758e0, 0xc420848000, 0x16, 0x169e020, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).ServeRequest(0xc420fa7ae0, 0x1d72340, 0xc420de0240, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d0e0, 0xc4201da000, 0xc420fa7ae0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 237 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0e470, 0x72, 0xc421894ce8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc421d33198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc421d33198, 0xc4220d6000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc421d33180, 0xc4220d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc421d33180, 0xc4220d6000, 0x1000, 0x1000, 0xc421894dd8, 0x0, 0xc421894e08) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc4204f62b0, 0xc4220d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Read(0xc421bc07e0, 0xc422a913a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadAtLeast(0x1d59aa0, 0xc421bc07e0, 0xc422a913a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadFull(0x1d59aa0, 0xc421bc07e0, 0xc422a913a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422558000, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422558000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 12 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/http.(*persistConn).writeLoop(0xc42042a120) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by net/http.(*Transport).dialConn Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 120 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0e7b0, 0x72, 0xc42105bce8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420991818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420991818, 0xc420f44000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc420991800, 0xc420f44000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc420991800, 0xc420f44000, 0x1000, 0x1000, 0xc4203a65d8, 0x1, 0xc4203a6608) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc42020c848, 0xc420f44000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Read(0xc4212b6180, 0xc4212768c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadAtLeast(0x1d59aa0, 0xc4212b6180, 0xc4212768c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadFull(0x1d59aa0, 0xc4212b6180, 0xc4212768c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420232210, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420232210) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 219 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b4000, 0xc4201daa80, 0xc421c602a0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc42122c2e0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 146 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200b4000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc42051aaf0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 112 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0e610, 0x72, 0xc421056ce8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420f76118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420f76118, 0xc4206ef000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc420f76100, 0xc4206ef000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc420f76100, 0xc4206ef000, 0x1000, 0x1000, 0xc421056dd8, 0x1, 0xc421056e08) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc42000e740, 0xc4206ef000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Read(0xc421119da0, 0xc421248fc0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadAtLeast(0x1d59aa0, 0xc421119da0, 0xc421248fc0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadFull(0x1d59aa0, 0xc421119da0, 0xc421248fc0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200be2c0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200be2c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 59 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420207090, 0x1c636a0, 0xc4202e29a0, 0x1d71840, 0xc4203ac0c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4202e29a0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 113 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200be2c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 122 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420232210) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 129 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420dc4000, 0xc42013d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).fill(0xc420b9c5a0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).ReadByte(0xc420b9c5a0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420bc6720, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42068fc60) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420b9c600, 0x16e4b60, 0xc42068fcc0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420b9c600, 0x16e4b60, 0xc42068fcc0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420967dc0, 0x16e4b60, 0xc42068fcc0, 0xc42068fcc0, 0xc420fa7b08) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420967dc0, 0xc42068fcc0, 0x2d, 0xc4207cc380) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequestHeader(0xc420fa7ae0, 0x1d72340, 0xc420967dc0, 0x60, 0x2, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequest(0xc420fa7ae0, 0x1d72340, 0xc420967dc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).ServeRequest(0xc420fa7ae0, 0x1d72340, 0xc420967dc0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d0e0, 0xc420dc4000, 0xc420fa7ae0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 194 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200be2c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 252 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4228350c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 253 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b4000, 0xc4201dab40, 0x1d6eb80, 0xc4228350c0, 0xc422db5080, 0xc422db50e0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc422835100) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 256 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc420206e60, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc420494780) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 241 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b4000, 0xc4201dab40, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b4000, 0xc4201dab40) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4222e91e0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 147 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200b4000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc42051ab00) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 148 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc421273230) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 149 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4201840d0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 150 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0ed60, 0x72, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420449f98, 0x72, 0xc4201fc500, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420449f98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Accept(0xc420449f80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).accept(0xc420449f80, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*TCPListener).accept(0xc42000e640, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*TCPListener).AcceptTCP(0xc42000e640, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420ad63c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 151 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0ec90, 0x72, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc4204fe098, 0x72, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc4204fe098, 0xc422f4c000, 0x10000, 0x10000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).ReadFrom(0xc4204fe080, 0xc422f4c000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).readFrom(0xc4204fe080, 0xc422f4c000, 0x10000, 0x10000, 0xc422f4c000, 0x10000, 0xc4205dde60, 0x4126b9, 0xc4204a8000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*UDPConn).readFrom(0xc42000e660, 0xc422f4c000, 0x10000, 0x10000, 0x101, 0xc4205dde18, 0xc4205dde90, 0x443cf7) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*UDPConn).ReadFrom(0xc42000e660, 0xc422f4c000, 0x10000, 0x10000, 0x10000, 0x10000, 0x8d92, 0x7a535d8d, 0x2b96de0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420ad63c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 152 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420ad63c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 153 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420ad63c0, 0x12a05f200, 0xc4210daf00, 0xc420f46cc0, 0xc4204f8c00) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 154 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420ad63c0, 0xc420f46cc0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 155 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420ad63c0, 0x1dcd6500, 0xc4210daf60, 0xc420f46cc0, 0xc4204f8c10) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 156 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4202efe00) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 157 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4202efe00) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 158 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202efe00, 0x1bdd09a, 0x6, 0xc421273380) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 159 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202efe00, 0x1bdba6c, 0x5, 0xc4212733b0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 160 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202efe00, 0x1bdbc6a, 0x5, 0xc4212733e0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 162 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4210dafc0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4210dafc0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4210dafc0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 163 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4210db020, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4210db020, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0xd40010) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4210db020) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 164 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4210db080, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4210db080, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4210db080) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 165 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4210db0e0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4210db0e0, 0x1dcd6500, 0x7f20d8ea2c98, 0xc420360c20, 0x0, 0x0, 0xc420b10960) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4210db0e0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 166 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0e3a0, 0x72, 0xc420d15490) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc422576718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc422576718, 0xc42237f000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc422576700, 0xc42237f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc422576700, 0xc42237f000, 0x1000, 0x1000, 0x1, 0xc42237f000, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc4204f6c38, 0xc42237f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Read(0xc4211b6480, 0xc4220fb164, 0x1, 0x1, 0x1000, 0x1000, 0xc420d15698) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).readTCP(0xc420ad63c0, 0x1d7d320, 0xc4204f6c38, 0x2b96de0, 0x0, 0x0, 0x2b96de0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:912 +0xee Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc420ad63c0, 0xc421f03810, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:772 +0x357 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc420ad63c0, 0xc421f03810, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc420ad63c0, 0xc420328400, 0x13, 0x20, 0x3, 0x3, 0xc4211ec560) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc4202efe00, 0xc420328400, 0x13, 0x20, 0xc4211ec501, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc420494780, 0xc420328400, 0x13, 0x20, 0xc4211ec560, 0x10, 0x10) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc4203a9740, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc42105f680, 0xc420494780) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 167 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc420494780) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 168 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc420494780) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 169 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0ee30, 0x72, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420449b18, 0x72, 0xc420fb0100, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420449b18, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Accept(0xc420449b00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).accept(0xc420449b00, 0x1, 0x1, 0xc42104bf20) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*TCPListener).accept(0xc42000f1f8, 0x406b4a, 0xc420f34300, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*TCPListener).Accept(0xc42000f1f8, 0xc420f34300, 0x0, 0x3f800000, 0xc421464200) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc420494780, 0x1d71840, 0xc421464200) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 170 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4204f00a0, 0x3b9aca00, 0xc42044e3c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 171 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc420206e60, 0x3b9aca00, 0xc42044e3c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 172 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc42035a480, 0x3b9aca00, 0xc42044e3c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 173 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc42021be10, 0x3b9aca00, 0xc42044e3c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 174 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc420494780) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 175 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0ebc0, 0x72, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420dae718, 0x72, 0xc420cac000, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420dae718, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Accept(0xc420dae700, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).accept(0xc420dae700, 0x412d88, 0x30, 0x1a1fac0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*TCPListener).accept(0xc420202ab0, 0x5fea4a, 0x1a1fac0, 0xc4212721b0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*TCPListener).Accept(0xc420202ab0, 0xc42003e028, 0x1830320, 0x2ab0680, 0x1b7b4c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/http.(*Server).Serve(0xc420a66410, 0x1d6fc00, 0xc420202ab0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/http.Serve(0x1d6fc00, 0xc420202ab0, 0x1d5df00, 0xc4212720f0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc4214642c0, 0x1d6fc00, 0xc420202ab0, 0xc4204f9560, 0xc42105f740) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 176 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420dae800, 0xc4204f9580, 0xc420f474a0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 177 [sleep]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: time.Sleep(0x29d301c2d) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc42031a5b0, 0xc420dae800) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 135 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202e2a50, 0xc4205dad10, 0x30, 0xf9866a) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202e2a50, 0x1c61988, 0xc420494780, 0x1d71840, 0xc421464200) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc42020c030, 0xc4210e83c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc42020c030, 0xc4210e83c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 136 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0e950, 0x72, 0xc4205dcce8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420c14398, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420c14398, 0xc421042000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc420c14380, 0xc421042000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc420c14380, 0xc421042000, 0x1000, 0x1000, 0xc4205dcdd8, 0x1, 0xc4205dce08) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc42020c030, 0xc421042000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Read(0xc421268b40, 0xc421248140, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadAtLeast(0x1d59aa0, 0xc421268b40, 0xc421248140, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadFull(0x1d59aa0, 0xc421268b40, 0xc421248140, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202e2a50, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202e2a50) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 137 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202e2a50) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 138 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202e2a50) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 255 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc421394780) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 251 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4201db2c0, 0xc42203a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).fill(0xc42103dec0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).ReadByte(0xc42103dec0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42274c240, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4220f0860) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42103df20, 0x16e4b60, 0xc4220f08c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42103df20, 0x16e4b60, 0xc4220f08c0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422834bc0, 0x16e4b60, 0xc4220f08c0, 0xc4220f08c0, 0xc42288f4c8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422834bc0, 0xc4220f08c0, 0x2d, 0xc421ebd500) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequestHeader(0xc42288f4a0, 0x1d72340, 0xc422834bc0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a8fa0, 0x8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequest(0xc42288f4a0, 0x1d72340, 0xc422834bc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).ServeRequest(0xc42288f4a0, 0x1d72340, 0xc422834bc0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d0e0, 0xc4201db2c0, 0xc42288f4a0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 242 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422558fd0, 0xc420f8ad10, 0x30, 0xf9866a) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422558fd0, 0x1c61988, 0xc420494780, 0x1d71840, 0xc421464200) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc4204f6890, 0xc42288f400) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc4204f6890, 0xc42288f400) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 142 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42035e180, 0xc42098e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).fill(0xc4211339e0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).ReadByte(0xc4211339e0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42114ac90, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4210e42c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421133a40, 0x16e4b60, 0xc4210e4320) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421133a40, 0x16e4b60, 0xc4210e4320, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421152c40, 0x16e4b60, 0xc4210e4320, 0xc4210e4320, 0xc4210e8488) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421152c40, 0xc4210e4320, 0x2d, 0xc420958820) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequestHeader(0xc4210e8460, 0x1d72340, 0xc421152c40, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a8fa0, 0x8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequest(0xc4210e8460, 0x1d72340, 0xc421152c40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).ServeRequest(0xc4210e8460, 0x1d72340, 0xc421152c40, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d0e0, 0xc42035e180, 0xc4210e8460) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 240 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b4000, 0xc4201daa80) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:123 +0x1fd Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4222e91c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 111 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200be2c0, 0xc421046d10, 0x30, 0xf9866a) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200be2c0, 0x1c61988, 0xc420494780, 0x1d71840, 0xc421464200) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc42000e740, 0xc420fa7a40) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc42000e740, 0xc420fa7a40) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 197 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420dc4780, 0xc4213ee000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).fill(0xc420f96ba0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).ReadByte(0xc420f96ba0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42122ff80, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207866a0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420f96c00, 0x16e4b60, 0xc420786720) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420f96c00, 0x16e4b60, 0xc420786720, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420f94840, 0x16e4b60, 0xc420786720, 0xc420786720, 0xc421256e38) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420f94840, 0xc420786720, 0x2d, 0xc420959520) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequestHeader(0xc421256e10, 0x1d72340, 0xc420f94840, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200a8fa0, 0x8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequest(0xc421256e10, 0x1d72340, 0xc420f94840, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).ServeRequest(0xc421256e10, 0x1d72340, 0xc420f94840, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d0e0, 0xc420dc4780, 0xc421256e10) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 229 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421a4e000, 0xc420022d10, 0x30, 0xf9866a) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421a4e000, 0x1c61988, 0xc420494780, 0x1d71840, 0xc421464200) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc42020c010, 0xc420faa050) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc42020c010, 0xc420faa050) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 210 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202322c0, 0xc42105ad10, 0x30, 0xf9866a) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202322c0, 0x1c61988, 0xc420494780, 0x1d71840, 0xc421464200) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc4204f6ea8, 0xc421256d70) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc4204f6ea8, 0xc421256d70) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 211 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0e540, 0x72, 0xc420f87ce8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420f5f898, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420f5f898, 0xc4210cf000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc420f5f880, 0xc4210cf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc420f5f880, 0xc4210cf000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc4204f6ea8, 0xc4210cf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Read(0xc4214f8c00, 0xc420789390, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadAtLeast(0x1d59aa0, 0xc4214f8c00, 0xc420789390, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadFull(0x1d59aa0, 0xc4214f8c00, 0xc420789390, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202322c0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202322c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 212 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202322c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 213 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202322c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 214 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420f96ae0, 0xc420bb5700, 0x20, 0x20, 0x169d4e0, 0xc420662600) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4210e3aa0, 0x1d718c0, 0xc420f96ae0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420494780, 0xc420bb5950, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4210989c0, 0xc420fca000, 0xc42122fcb0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: reflect.Value.call(0xc42019ade0, 0xc4204f71c8, 0x13, 0x1bda9a0, 0x4, 0xc421055d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: reflect.Value.Call(0xc42019ade0, 0xc4204f71c8, 0x13, 0xc421055d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*service).call(0xc420ee51c0, 0xc421256e10, 0xc420afa298, 0x0, 0xc4214d8980, 0xc4206f08a0, 0x19758e0, 0xc420fca000, 0x16, 0x169e020, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).ServeRequest(0xc421256e10, 0x1d72340, 0xc420ee5200, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d0e0, 0xc420dc46c0, 0xc421256e10) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 198 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202e28f0, 0xc420f86d10, 0x30, 0xf9866a) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202e28f0, 0x1c61988, 0xc420494780, 0x1d71840, 0xc421464200) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc4202021b0, 0xc4217542d0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d320, 0xc4202021b0, 0xc4217542d0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 199 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0e2d0, 0x72, 0xc420f85ce8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420aa2198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420aa2198, 0xc420eb6000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc420aa2180, 0xc420eb6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc420aa2180, 0xc420eb6000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc4202021b0, 0xc420eb6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Read(0xc421781200, 0xc4210ea3f0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadAtLeast(0x1d59aa0, 0xc421781200, 0xc4210ea3f0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadFull(0x1d59aa0, 0xc421781200, 0xc4210ea3f0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202e28f0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202e28f0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 200 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202e28f0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 201 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202e28f0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 202 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420c53da0, 0xc42177b700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420d9cde0, 0x1d718c0, 0xc420c53da0, 0x0, 0xc420c53d00) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420494780, 0xc42177b950, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42145b1a0, 0xc420f76000, 0xc420d9c930, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: reflect.Value.call(0xc42019ade0, 0xc420202c80, 0x13, 0x1bda9a0, 0x4, 0xc42177bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: reflect.Value.Call(0xc42019ade0, 0xc420202c80, 0x13, 0xc42177bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*service).call(0xc42178c900, 0xc421754370, 0xc421788818, 0x0, 0xc420c15180, 0xc42133e520, 0x19758e0, 0xc420f76000, 0x16, 0x169e020, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).ServeRequest(0xc421754370, 0x1d72340, 0xc42178c940, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d0e0, 0xc420fe2000, 0xc421754370) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 230 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0e200, 0x72, 0xc421892ce8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc420fca198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc420fca198, 0xc4204cb000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc420fca180, 0xc4204cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc420fca180, 0xc4204cb000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc42020c010, 0xc4204cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Read(0xc421632060, 0xc421d54650, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadAtLeast(0x1d59aa0, 0xc421632060, 0xc421d54650, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadFull(0x1d59aa0, 0xc421632060, 0xc421d54650, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421a4e000, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421a4e000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 231 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421a4e000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 232 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421a4e000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 233 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4201f4840, 0xc420d19700, 0x20, 0x20, 0x169d4e0, 0xc420662600) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421341e60, 0x1d718c0, 0xc4201f4840, 0x0, 0xc4201f4800) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420494780, 0xc420d19950, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4210d62a0, 0xc420aa2500, 0xc421341c20, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: reflect.Value.call(0xc42019ade0, 0xc42020cb78, 0x13, 0x1bda9a0, 0x4, 0xc420d19d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: reflect.Value.Call(0xc42019ade0, 0xc42020cb78, 0x13, 0xc420d19d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*service).call(0xc42146c7c0, 0xc420faa0f0, 0xc421b535c8, 0x0, 0xc4214d9900, 0xc4210dc0c0, 0x19758e0, 0xc420aa2500, 0x16, 0x169e020, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).ServeRequest(0xc420faa0f0, 0x1d72340, 0xc42146c800, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d0e0, 0xc420fe20c0, 0xc420faa0f0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 234 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4201da300, 0xc4215c6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).fill(0xc4201f4720) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).ReadByte(0xc4201f4720, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421341bc0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4210dd8a0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4201f4780, 0x16e4b60, 0xc4210dd900) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4201f4780, 0x16e4b60, 0xc4210dd900, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421572c00, 0x16e4b60, 0xc4210dd900, 0xc4210dd900, 0xc420faa118) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421572c00, 0xc4210dd900, 0x2d, 0xc421b166a0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequestHeader(0xc420faa0f0, 0x1d72340, 0xc421572c00, 0x30, 0x1, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).readRequest(0xc420faa0f0, 0x1d72340, 0xc421572c00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net/rpc.(*Server).ServeRequest(0xc420faa0f0, 0x1d72340, 0xc421572c00, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420494780, 0x1d71840, 0xc421464200, 0x1d7d0e0, 0xc4201da300, 0xc420faa0f0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 187 [runnable]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4204f00a0, 0xc420709e40, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:225 +0x33e Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4204f00a0, 0xc420709e40) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc420494780, 0xc420533201, 0xc420533200) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc420494780, 0xc4229fc420, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc420494780, 0xc4229fc420) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc421248090, 0xc420494780, 0xc4229fc420) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 238 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422558000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 239 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422558000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 221 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b4000, 0xc4201dab40, 0xc421c60420) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc42122c360) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 243 [IO wait]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.runtime_pollWait(0x7f20d8f0ea20, 0x72, 0xc4205dbce8) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).wait(0xc4226d4718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*pollDesc).waitRead(0xc4226d4718, 0xc4228cb000, 0x1000, 0x1000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: internal/poll.(*FD).Read(0xc4226d4700, 0xc4228cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*netFD).Read(0xc4226d4700, 0xc4228cb000, 0x1000, 0x1000, 0xa8, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: net.(*conn).Read(0xc4204f6890, 0xc4228cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: bufio.(*Reader).Read(0xc421bc1980, 0xc421cbd230, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadAtLeast(0x1d59aa0, 0xc421bc1980, 0xc421cbd230, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: io.ReadFull(0x1d59aa0, 0xc421bc1980, 0xc421cbd230, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422558fd0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422558fd0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 244 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422558fd0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 245 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422558fd0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 274 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc42035a480) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 225 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc422b14000) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 254 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc421394780) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 275 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc42035a480) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 276 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc42274d560) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 277 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422835340, 0xc421895ce8, 0x20, 0x20, 0x18, 0xc42122ca40) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42200b8f0, 0x1d71840, 0xc422835340, 0xc42122ca40, 0x184987) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420533200, 0xc4215dded0, 0x184987, 0x1d71840, 0xc422835340, 0xc4204fe560, 0xc42274d4d0, 0xc420b964b0, 0x24, 0x2bba400) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc4204fe500, 0x1d71840, 0xc422835340, 0x184987, 0xc422a75f80, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc4204fe500, 0x1d71840, 0xc422835340) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 278 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4228353c0, 0xc4215d7770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224369c0, 0x1d71840, 0xc4228353c0, 0x0, 0x1849b5) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420533200, 0xc4215d7960, 0x1849b5, 0x1d71840, 0xc4228353c0, 0x552bde, 0xc4215d7950, 0xc4215d7970, 0xf33ba5, 0xc42193f088) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc42193f030, 0x1d71840, 0xc4228353c0, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc42193f030) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 279 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422835380, 0xc422b49a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42200bd10, 0x1d71840, 0xc422835380, 0xc42200be00, 0x1833a4) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420533200, 0xc422b49c68, 0x1833a4, 0x1d71840, 0xc422835380, 0x552bde, 0xc420206be0, 0x2, 0xc4210edd40, 0x40) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422835400, 0x1833a4, 0x3b, 0xc421890de8, 0x1, 0x1) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422835400) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 280 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422835540) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 281 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4214660f0, 0x1d71840, 0xc422835380) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 262 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422436720, 0xc4229fcde0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 263 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422436720, 0xc4229fce40) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: goroutine 264 [select]: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422b84040, 0xc422925d28, 0x20, 0x20, 0xc4204a8000, 0x1bd9a98) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224368d0, 0x1d71840, 0xc422b84040, 0xc4223378c0, 0x1849b5) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420533200, 0xc42230df20, 0x1849b5, 0x1d71840, 0xc422b84040, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc422b14000, 0x1849b5, 0xc4203a87b8, 0xc4203a86e8, 0xc42229be60, 0xf959ca, 0x1, 0x0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc422b14000, 0x1849b5, 0xc4221c83c0) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25655]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:27:33 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:27:33 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:27:33 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: ==> Starting Nomad agent... Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: ==> Nomad agent configuration: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: Client: false Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: Log Level: INFO Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: Region: amrv01 (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: Server: true Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: Version: 0.8.4 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.478186 [INFO] nomad: starting 4 scheduling worker(s) for [batch system service _core] Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.478704 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: Attempting re-join to previously known node: consulnomad-03.aaf01: 172.26.1.78:4648 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [WARN] memberlist: Refuting an alive message Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659467 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659510 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659526 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659538 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659550 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659561 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659724 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659739 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659751 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659799 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659812 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659907 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.659926 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.660059 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.660105 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.660129 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.660140 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33 [INFO] serf: Re-joined to previously known node: consulnomad-03.aaf01: 172.26.1.78:4648 Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.660258 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:33 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:33.660271 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:34 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:34 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:35 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:35.374972 [ERR] http: Request /v1/jobs?prefix=bobrovnik-a-jupiternoteBook®ion=amrv01, error: rpc error: failed to get conn: dial tcp 172.24.10.89:4647: connect: connection refused Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 34 Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36.820219 [INFO] nomad: cluster leadership acquired Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:36 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:36 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:37 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:37 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:37 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:37 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: 2018/08/23 15:27:37 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: fatal error: concurrent map writes Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 240 [running]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc422b58d08 sp=0xc422b58ce8 pc=0x42d161 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: runtime.mapdelete(0x1844be0, 0xc42036be00, 0xc422b58e20) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc422b58d68 sp=0xc422b58d08 pc=0x40a761 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc4202c78c0, 0x1d6dd40, 0xc42020e0e8, 0x0, 0xed3109f39) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc422b58e50 sp=0xc422b58d68 pc=0xf2164c Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4200c6140, 0x1d71840, 0xc422ec8640) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc422b58fc8 sp=0xc422b58e50 pc=0xf56bb7 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: runtime.goexit() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc422b58fd0 sp=0xc422b58fc8 pc=0x45c711 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 1 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42036ed20, 0xc4200ef400) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42036ed20, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc42049cf00, 0xc42049cf00, 0xc420432100, 0x37) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204d9f68) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: main.main() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 5 [syscall]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: os/signal.signal_recv(0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: os/signal.loop() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by os/signal.init.0 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 172 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205680b0, 0xc4211bfd10, 0x30, 0xf9866a) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205680b0, 0x1c61988, 0xc42039a000, 0x1d71840, 0xc420f51f00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d320, 0xc4203f8278, 0xc42134e1e0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d320, 0xc4203f8278, 0xc42134e1e0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 28 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463f00, 0x72, 0xc4200639a8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc420134298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc420134298, 0xc4203ea000, 0x1000, 0x1000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Read(0xc420134280, 0xc4203ea000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).Read(0xc420134280, 0xc4203ea000, 0x1000, 0x1000, 0x458600, 0xc42056a180, 0x4) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*conn).Read(0xc42000e008, 0xc4203ea000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/http.(*persistConn).Read(0xc4204fa000, 0xc4203ea000, 0x1000, 0x1000, 0xc420063b98, 0x405675, 0xc420090600) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).fill(0xc420196240) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).Peek(0xc420196240, 0x1, 0x0, 0x0, 0x1, 0xc420f117a0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/http.(*persistConn).readLoop(0xc4204fa000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by net/http.(*Transport).dialConn Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 9 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc420401280) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 52 [select, locked to thread]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: runtime.selectgo(0xc420264f50, 0xc420090360) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: runtime.ensureSigM.func1() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: runtime.goexit() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 10 [sleep]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: time.Sleep(0x3b9aca00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420568000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 11 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420574300) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 30 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4201ff180, 0x1d71840, 0xc4201ff200) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 41 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc42038a060) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 239 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc420201310, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc42039a000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 43 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4204240d0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc420566300, 0xc4201fc350) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 29 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/http.(*persistConn).writeLoop(0xc4204fa000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by net/http.(*Transport).dialConn Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 106 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc42031e000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42031e000, 0xc4210187b0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 185 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42238c000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 163 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/http.(*persistConn).writeLoop(0xc4214eb440) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by net/http.(*Transport).dialConn Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 243 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc42031e000, 0xc4217826c0, 0xc422a58c00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:37 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42031e000, 0xc420540180) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 189 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42238c210) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 190 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42238c210) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 208 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42196d130, 0xc42168cd10, 0x30, 0xf9866a) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42196d130, 0x1c61988, 0xc42039a000, 0x1d71840, 0xc420f51f00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d320, 0xc4202047e0, 0xc42252e820) Aug 23 15:27:37 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d320, 0xc4202047e0, 0xc42252e820) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 44 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4203aa230, 0x1c636a0, 0xc4204024d0, 0x1d71840, 0xc420401900) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4204024d0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 105 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc42031e000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc42031e000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc42031e000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42031e000, 0xc4210187a0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:37 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 109 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc42081a5b0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 191 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421c31620, 0xc4229af700, 0x20, 0x20, 0x169d4e0, 0xc420918720) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421fc46f0, 0x1d718c0, 0xc421c31620, 0x0, 0xc421c31600) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42039a000, 0xc4229af950, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42143da40, 0xc420963b00, 0xc421fc43c0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: reflect.Value.call(0xc4202feae0, 0xc42020ec88, 0x13, 0x1bda9a0, 0x4, 0xc4229afd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: reflect.Value.Call(0xc4202feae0, 0xc42020ec88, 0x13, 0xc4229afd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*service).call(0xc4226dc4c0, 0xc4213550e0, 0xc422cd5dc8, 0x0, 0xc422df6500, 0xc4216d2e00, 0x19758e0, 0xc420963b00, 0x16, 0x169e020, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).ServeRequest(0xc4213550e0, 0x1d72340, 0xc4226dc540, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d0e0, 0xc42250a0c0, 0xc4213550e0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 258 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42196d130) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 107 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc42031e000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42031e000, 0xc4210187c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 120 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463bc0, 0x72, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc420e56718, 0x72, 0xc420083600, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc420e56718, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Accept(0xc420e56700, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).accept(0xc420e56700, 0x402037, 0xc422d48080, 0x1805ec0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*TCPListener).accept(0xc42000e610, 0x458e40, 0xc4204bae90, 0xc4204bae98) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*TCPListener).Accept(0xc42000e610, 0x1c65af8, 0xc422d48000, 0x1d71900, 0xc420fa8120) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/http.(*Server).Serve(0xc4208c8f70, 0x1d6fc00, 0xc42000e610, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/http.Serve(0x1d6fc00, 0xc42000e610, 0x1d5df00, 0xc420fa8060, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420fa2080, 0x1d6fc00, 0xc42000e610, 0xc4210183c0, 0xc420f9e090) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 108 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420f638f0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 110 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463d60, 0x72, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc42098c198, 0x72, 0xc420431400, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc42098c198, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Accept(0xc42098c180, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).accept(0xc42098c180, 0xc4203f9bc0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*TCPListener).accept(0xc42000e5b0, 0x458e40, 0xc420050790, 0xc420050798) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*TCPListener).AcceptTCP(0xc42000e5b0, 0x1c63658, 0xc4205763c0, 0xc4203f9bc0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4205763c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 111 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463c90, 0x72, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc42098c318, 0x72, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc42098c318, 0xc423366000, 0x10000, 0x10000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).ReadFrom(0xc42098c300, 0xc423366000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).readFrom(0xc42098c300, 0xc423366000, 0x10000, 0x10000, 0x1, 0x0, 0xc421690e60, 0x41274d, 0xc423366000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*UDPConn).readFrom(0xc42000e5e0, 0xc423366000, 0x10000, 0x10000, 0x101, 0xc421690e18, 0xc421690e90, 0x443cf7) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*UDPConn).ReadFrom(0xc42000e5e0, 0xc423366000, 0x10000, 0x10000, 0x10000, 0x10000, 0x6039, 0xe49bad09, 0x2b96de0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4205763c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 112 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4205763c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 113 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4205763c0, 0x12a05f200, 0xc421316d20, 0xc420effd40, 0xc421018cd0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:110 +0xef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 130 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4205763c0, 0xc420effd40) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 131 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4205763c0, 0x1dcd6500, 0xc421316d80, 0xc420effd40, 0xc421018cf0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 132 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4202f1e00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 133 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4202f1e00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 134 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202f1e00, 0x1bdd09a, 0x6, 0xc420f63a40) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 135 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202f1e00, 0x1bdba6c, 0x5, 0xc420f63a70) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 136 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202f1e00, 0x1bdbc6a, 0x5, 0xc420f63aa0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 188 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e4637b0, 0x72, 0xc4204bfce8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc420134d18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc420134d18, 0xc42209b000, 0x1000, 0x1000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Read(0xc420134d00, 0xc42209b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).Read(0xc420134d00, 0xc42209b000, 0x1000, 0x1000, 0xc4204bfdd8, 0x1, 0xc4204bfe08) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*conn).Read(0xc42020e1e0, 0xc42209b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).Read(0xc4211f3980, 0xc4229e4770, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadAtLeast(0x1d59aa0, 0xc4211f3980, 0xc4229e4770, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadFull(0x1d59aa0, 0xc4211f3980, 0xc4229e4770, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42238c210, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42238c210) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 138 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421316de0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421316de0, 0x1dcd6500, 0xc420f11860, 0x9ebabe, 0x1, 0x0, 0x435248) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421316de0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 139 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421316e40, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421316e40, 0x1dcd6500, 0x1bd9928, 0x2, 0x1bdab30, 0x4, 0xc420da8e00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421316e40) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 140 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421316ea0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421316ea0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421316ea0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 141 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421316f00, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421316f00, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421316f00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 142 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463200, 0x77, 0xc42102b800) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc42254b418, 0x77, 0xc42135d000, 0xc42135d028, 0x436544) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitWrite(0xc42254b418, 0xc42135d000, 0x28, 0x611b26) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).WaitWrite(0xc42254b400, 0x1c65988, 0x1d718c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:440 +0x37 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).connect(0xc42254b400, 0x1d718c0, 0xc422d1b200, 0x0, 0x0, 0x1d5c800, 0xc422aef580, 0x0, 0x0, 0x0, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:152 +0x299 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).dial(0xc42254b400, 0x1d718c0, 0xc422d1b200, 0x1d7a200, 0x0, 0x1d7a200, 0xc422e42870, 0xc42135d248, 0x62f2be) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/sock_posix.go:142 +0xe9 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.socket(0x1d718c0, 0xc422d1b200, 0x1bda392, 0x3, 0x2, 0x1, 0x0, 0x0, 0x1d7a200, 0x0, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/sock_posix.go:93 +0x1a6 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.internetSocket(0x1d718c0, 0xc422d1b200, 0x1bda392, 0x3, 0x1d7a200, 0x0, 0x1d7a200, 0xc422e42870, 0x1, 0x0, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/ipsock_posix.go:141 +0x129 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.doDialTCP(0x1d718c0, 0xc422d1b200, 0x1bda392, 0x3, 0x0, 0xc422e42870, 0x2bb8aa0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/tcpsock_posix.go:62 +0xb9 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.dialTCP(0x1d718c0, 0xc422d1b200, 0x1bda392, 0x3, 0x0, 0xc422e42870, 0x7e4ca091a, 0x2b96de0, 0x6fc2390ea) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/tcpsock_posix.go:58 +0xe4 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.dialSingle(0x1d718c0, 0xc422d1b200, 0xc4222e6000, 0x1d62b00, 0xc422e42870, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/dial.go:547 +0x375 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.dialSerial(0x1d718c0, 0xc422d1b200, 0xc4222e6000, 0xc420725180, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/dial.go:515 +0x22d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*Dialer).DialContext(0xc42135d848, 0x1d71880, 0xc42003e028, 0x1bda392, 0x3, 0xc422aef560, 0x11, 0x0, 0x0, 0x0, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/dial.go:397 +0x678 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*Dialer).Dial(0xc42135d848, 0x1bda392, 0x3, 0xc422aef560, 0x11, 0x20, 0x20, 0x17503e0, 0x2ade3c8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/dial.go:320 +0x75 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc4205763c0, 0xc422d23900, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:758 +0x14e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc4205763c0, 0xc422d23900, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc4205763c0, 0xc420448000, 0x13, 0x20, 0x3, 0x3, 0xc4208ca220) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc4202f1e00, 0xc420448000, 0x13, 0x20, 0xc4208ca201, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc42039a000, 0xc420448000, 0x13, 0x20, 0xc4208ca220, 0x10, 0x10) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc42056ff40, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc421395aa0, 0xc42039a000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 143 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc42039a000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 144 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc42039a000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 145 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463e30, 0x72, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc42046d318, 0x72, 0xc420212f00, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc42046d318, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Accept(0xc42046d300, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).accept(0xc42046d300, 0x1, 0x1, 0xc4201aaf20) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*TCPListener).accept(0xc4203f8d30, 0x406b4a, 0xc420aa7440, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*TCPListener).Accept(0xc4203f8d30, 0xc420aa7440, 0x0, 0x3f800000, 0xc420f51f00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc42039a000, 0x1d71840, 0xc420f51f00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 146 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4200c6140, 0x3b9aca00, 0xc4203a60c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 147 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc420201310, 0x3b9aca00, 0xc4203a60c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 148 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420578120, 0x3b9aca00, 0xc4203a60c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 149 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4204240d0, 0x3b9aca00, 0xc4203a60c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 150 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc42039a000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 121 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420e56b00, 0xc4210183d0, 0xc420edd500) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 122 [sleep]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: time.Sleep(0x44d5bf367) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc42036ed20, 0xc420e56b00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 241 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420578120) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 162 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463a20, 0x72, 0xc4201ab9a8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc4209acc98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc4209acc98, 0xc4211f7000, 0x1000, 0x1000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Read(0xc4209acc80, 0xc4211f7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).Read(0xc4209acc80, 0xc4211f7000, 0x1000, 0x1000, 0x458600, 0xc42102b800, 0x4) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*conn).Read(0xc4203f8328, 0xc4211f7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/http.(*persistConn).Read(0xc4214eb440, 0xc4211f7000, 0x1000, 0x1000, 0xc4201abb98, 0x405675, 0xc420efe2a0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).fill(0xc421200c00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).Peek(0xc421200c00, 0x1, 0x0, 0x0, 0x1, 0xc420831920, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/http.(*persistConn).readLoop(0xc4214eb440) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by net/http.(*Transport).dialConn Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 238 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4206e3900) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 199 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421ac6f20) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 197 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421ac6f20, 0xc4211c1d10, 0x30, 0xf9866a) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421ac6f20, 0x1c61988, 0xc42039a000, 0x1d71840, 0xc420f51f00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d320, 0xc4202043c0, 0xc421d379f0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d320, 0xc4202043c0, 0xc421d379f0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 198 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463880, 0x72, 0xc4201acce8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc420881318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc420881318, 0xc4219f2000, 0x1000, 0x1000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Read(0xc420881300, 0xc4219f2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).Read(0xc420881300, 0xc4219f2000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*conn).Read(0xc4202043c0, 0xc4219f2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).Read(0xc4215c8000, 0xc4228c54a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadAtLeast(0x1d59aa0, 0xc4215c8000, 0xc4228c54a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadFull(0x1d59aa0, 0xc4215c8000, 0xc4228c54a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421ac6f20, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421ac6f20) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 291 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc422eb29c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 290 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420578120) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 231 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc42031e000, 0xc421782780, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc42031e000, 0xc421782780) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42031e000, 0xc420c66b80) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 187 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42238c210, 0xc421693d10, 0x30, 0xf9866a) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42238c210, 0x1c61988, 0xc42039a000, 0x1d71840, 0xc420f51f00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d320, 0xc42020e1e0, 0xc421355040) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d320, 0xc42020e1e0, 0xc421355040) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 173 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463610, 0x72, 0xc420eb6ce8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc420e56218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc420e56218, 0xc421359000, 0x1000, 0x1000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Read(0xc420e56200, 0xc421359000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).Read(0xc420e56200, 0xc421359000, 0x1000, 0x1000, 0xc420eb6dd8, 0x1, 0xc420eb6e08) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*conn).Read(0xc4203f8278, 0xc421359000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).Read(0xc421278960, 0xc42091a040, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadAtLeast(0x1d59aa0, 0xc421278960, 0xc42091a040, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadFull(0x1d59aa0, 0xc421278960, 0xc42091a040, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205680b0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205680b0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 174 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205680b0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 175 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205680b0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 176 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420574000, 0xc42071b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).fill(0xc4202ffbc0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).ReadByte(0xc4202ffbc0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420904ab0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42063d0c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4202ffc20, 0x16e4b60, 0xc42063d120) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4202ffc20, 0x16e4b60, 0xc42063d120, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4214faf40, 0x16e4b60, 0xc42063d120, 0xc42063d120, 0xc42134e2a8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4214faf40, 0xc42063d120, 0x2d, 0xc420a44140) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).readRequestHeader(0xc42134e280, 0x1d72340, 0xc4214faf40, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4203aaf00, 0x8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).readRequest(0xc42134e280, 0x1d72340, 0xc4214faf40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).ServeRequest(0xc42134e280, 0x1d72340, 0xc4214faf40, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d0e0, 0xc420574000, 0xc42134e280) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 230 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc42031e000, 0xc4217826c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:123 +0x1fd Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42031e000, 0xc420c66b60) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 205 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422684000, 0xc4201add10, 0x30, 0xf9866a) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422684000, 0x1c61988, 0xc42039a000, 0x1d71840, 0xc420f51f00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d320, 0xc4202047c8, 0xc42252e2d0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d320, 0xc4202047c8, 0xc42252e2d0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 184 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463950, 0x72, 0xc420061ce8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc42254b698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc42254b698, 0xc42235c000, 0x1000, 0x1000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Read(0xc42254b680, 0xc42235c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).Read(0xc42254b680, 0xc42235c000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*conn).Read(0xc42020e1c8, 0xc42235c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).Read(0xc4211f3260, 0xc4229e4440, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadAtLeast(0x1d59aa0, 0xc4211f3260, 0xc4229e4440, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadFull(0x1d59aa0, 0xc4211f3260, 0xc4229e4440, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42238c000, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42238c000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 201 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421c31da0, 0xc42135f700, 0x20, 0x20, 0x169d4e0, 0xc420918720) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42226c0f0, 0x1d718c0, 0xc421c31da0, 0x0, 0xc421c31d00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42039a000, 0xc42135f950, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4217a8180, 0xc420963d80, 0xc421fc5e90, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: reflect.Value.call(0xc4202feae0, 0xc420204760, 0x13, 0x1bda9a0, 0x4, 0xc42135fd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: reflect.Value.Call(0xc4202feae0, 0xc420204760, 0x13, 0xc42135fd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*service).call(0xc421163bc0, 0xc421d37a90, 0xc4226a1528, 0x0, 0xc42254af00, 0xc421492780, 0x19758e0, 0xc420963d80, 0x16, 0x169e020, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).ServeRequest(0xc421d37a90, 0x1d72340, 0xc421163c80, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d0e0, 0xc4204b9200, 0xc421d37a90) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 200 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421ac6f20) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 186 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42238c000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 216 [runnable]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4200c6140, 0xc4206f3600, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x20c Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4200c6140, 0xc4206f3600) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc42039a000, 0xc4203d3301, 0xc4203d3300) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc42039a000, 0xc422a44de0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc42039a000, 0xc422a44de0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420fda000, 0xc42039a000, 0xc422a44de0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 204 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc42031e000, 0xc421782780, 0xc422a14a80) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42031e000, 0xc421009820) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 209 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e4632d0, 0x72, 0xc420eb3ce8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc422040218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc422040218, 0xc42204c000, 0x1000, 0x1000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Read(0xc422040200, 0xc42204c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).Read(0xc422040200, 0xc42204c000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*conn).Read(0xc4202047e0, 0xc42204c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).Read(0xc421967200, 0xc422cd5ec0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadAtLeast(0x1d59aa0, 0xc421967200, 0xc422cd5ec0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadFull(0x1d59aa0, 0xc421967200, 0xc422cd5ec0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42196d130, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42196d130) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 219 [IO wait]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.runtime_pollWait(0x7f834e463540, 0x72, 0xc42168ece8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).wait(0xc42254bc18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*pollDesc).waitRead(0xc42254bc18, 0xc42216d000, 0x1000, 0x1000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: internal/poll.(*FD).Read(0xc42254bc00, 0xc42216d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).Read(0xc42254bc00, 0xc42216d000, 0x1000, 0x1000, 0xc42168edd8, 0x0, 0xc42168ee08) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*conn).Read(0xc4202047c8, 0xc42216d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).Read(0xc4217a9ce0, 0xc422c9e620, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadAtLeast(0x1d59aa0, 0xc4217a9ce0, 0xc422c9e620, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: io.ReadFull(0x1d59aa0, 0xc4217a9ce0, 0xc422c9e620, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422684000, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422684000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 220 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422684000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 221 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422684000) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 275 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net.(*netFD).connect.func2(0x1d718c0, 0xc422d1b200, 0xc42254b400, 0xc422ea0120, 0xc422ea00c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:129 +0xde Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by net.(*netFD).connect Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/fd_unix.go:128 +0x286 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 248 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420404d80, 0xc4227a8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).fill(0xc4212af9e0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).ReadByte(0xc4212af9e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421f498c0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4205409a0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4212afa40, 0x16e4b60, 0xc420540a00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4212afa40, 0x16e4b60, 0xc420540a00, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4221d8200, 0x16e4b60, 0xc420540a00, 0xc420540a00, 0xc421355108) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4221d8200, 0xc420540a00, 0x2d, 0xc421dd9b00) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).readRequestHeader(0xc4213550e0, 0x1d72340, 0xc4221d8200, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4203aaf00, 0x8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).readRequest(0xc4213550e0, 0x1d72340, 0xc4221d8200, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).ServeRequest(0xc4213550e0, 0x1d72340, 0xc4221d8200, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d0e0, 0xc420404d80, 0xc4213550e0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 259 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42196d130) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 260 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420404e40, 0xc42204d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).fill(0xc42258f740) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: bufio.(*Reader).ReadByte(0xc42258f740, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421fecb40, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206c0680) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42258f7a0, 0x16e4b60, 0xc4206c06e0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42258f7a0, 0x16e4b60, 0xc4206c06e0, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421608e00, 0x16e4b60, 0xc4206c06e0, 0xc4206c06e0, 0xc42252e8e8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421608e00, 0xc4206c06e0, 0x2d, 0xc422195680) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).readRequestHeader(0xc42252e8c0, 0x1d72340, 0xc421608e00, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4203aaf00, 0x8) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).readRequest(0xc42252e8c0, 0x1d72340, 0xc421608e00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: net/rpc.(*Server).ServeRequest(0xc42252e8c0, 0x1d72340, 0xc421608e00, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42039a000, 0x1d71840, 0xc420f51f00, 0x1d7d0e0, 0xc420404e40, 0xc42252e8c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 271 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc420f49b80, 0x7f834e3c9950, 0xc422e44140, 0x1, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:248 +0x2c1 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc420f49b80, 0x7f834e3c9950, 0xc422e44140, 0x7f834e3c9950, 0xc422e44140) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).latestEvalIndex(0xc42035eaa0, 0xc422ec7f88, 0xc422ec7ef8, 0x4) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:784 +0x89 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).handleAllocUpdate(0xc42035eaa0, 0xc42020e0c0, 0x1, 0x1, 0xe97194b6, 0x2b96de0, 0x440e95d84e, 0x0, 0xc4211b76d0, 0x43e2b2) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:514 +0x40 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc42035eaa0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:441 +0x591 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 237 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4206e3900) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 235 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc42031e000, 0xc421782780, 0x1d6eb80, 0xc422ec8080, 0xc422eb0180, 0xc422eb01e0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42031e000, 0xc422ec81c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 234 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc422ec8080) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 292 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422ec8680, 0xc422ec7ce8, 0x20, 0x20, 0x18, 0xc422ea2200) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422ece1b0, 0x1d71840, 0xc422ec8680, 0xc422ea2200, 0x184987) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203d3300, 0xc421361ed0, 0x184987, 0x1d71840, 0xc422ec8680, 0xc42046d660, 0xc422eb2990, 0xc420b6e8d0, 0x24, 0x2bba400) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc42046d600, 0x1d71840, 0xc422ec8680, 0x184987, 0xc422e44040, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc42046d600, 0x1d71840, 0xc422ec8680) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 293 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422ec8700, 0xc42124b770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422e180f0, 0x1d71840, 0xc422ec8700, 0x0, 0x1849b5) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203d3300, 0xc42124b960, 0x1849b5, 0x1d71840, 0xc422ec8700, 0x552bde, 0xc42124b950, 0xc42124b970, 0xf33ba5, 0xc421010678) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc421010620, 0x1d71840, 0xc422ec8700, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc421010620) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 294 [runnable]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: fmt.(*fmt).pad(0xc420404100, 0xc420404128, 0x26, 0x44) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/fmt/format.go:90 +0x3a9 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: fmt.(*fmt).fmt_q(0xc420404100, 0xc4203fb320, 0x24) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/fmt/format.go:420 +0x11f Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: fmt.(*pp).fmtString(0xc4204040c0, 0xc4203fb320, 0x24, 0xc400000071) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/fmt/print.go:446 +0x7c Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: fmt.(*pp).printArg(0xc4204040c0, 0x1734280, 0xc4205263c0, 0xc400000071) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/fmt/print.go:671 +0x789 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: fmt.(*pp).doPrintf(0xc4204040c0, 0x1c429c3, 0x4e, 0xc422ec4e48, 0x4, 0x4) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/fmt/print.go:1003 +0x157 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: fmt.Sprintf(0x1c429c3, 0x4e, 0xc422ec4e48, 0x4, 0x4, 0xc420bc9000, 0xc422cd4168) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/fmt/print.go:203 +0x66 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: log.(*Logger).Printf(0xc4202010e0, 0x1c429c3, 0x4e, 0xc422ec4e48, 0x4, 0x4) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /usr/local/go/src/log/log.go:179 +0x53 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422ec8780) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:197 +0x39d Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 295 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422ec87c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 296 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4205765a0, 0x1d71840, 0xc422ec86c0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 276 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422e428a0, 0xc422ea0180) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: goroutine 277 [select]: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422e428a0, 0xc422ea01e0) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25675]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:37 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:27:37 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:27:37 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: ==> Starting Nomad agent... Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: ==> Nomad agent configuration: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: Client: false Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: Log Level: INFO Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: Region: amrv01 (DC: test) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: Server: true Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: Version: 0.8.4 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:37 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:37 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:37 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:37 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:37.719498 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:37 [INFO] serf: Attempting re-join to previously known node: consulnomad-staging-02.asv01: 172.27.0.200:4648 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:37.719881 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:37 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:37 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:37.747064 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074223 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074249 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074261 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074287 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074300 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [WARN] memberlist: Refuting an alive message Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074561 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074580 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074598 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074612 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074888 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074906 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074917 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.074932 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38 [INFO] serf: Re-joined to previously known node: consulnomad-staging-02.asv01: 172.27.0.200:4648 Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.075213 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.075229 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.075241 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.075252 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:38 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:38.075263 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 35 Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39.490853 [INFO] nomad: cluster leadership acquired Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:39 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:40 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:40 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:40 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:40 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:40 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:40 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:40 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: 2018/08/23 15:27:40 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 1.217716085s Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: fatal error: concurrent map writes Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 223 [running]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc420572d08 sp=0xc420572ce8 pc=0x42d161 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: runtime.mapdelete(0x1844be0, 0xc4202632f0, 0xc420572e20) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc420572d68 sp=0xc420572d08 pc=0x40a761 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc420479cc0, 0x1d6dd40, 0xc420212f98, 0x0, 0xed3109f3c) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc420572e50 sp=0xc420572d68 pc=0xf2164c Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4200d40a0, 0x1d71840, 0xc4225e2940) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc420572fc8 sp=0xc420572e50 pc=0xf56bb7 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: runtime.goexit() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420572fd0 sp=0xc420572fc8 pc=0x45c711 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 1 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc4203517a0, 0xc4204997c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc4203517a0, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4203177c0, 0xc4203177c0, 0xc420287020, 0x37) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204d7f68) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc42009e058) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: main.main() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 5 [syscall]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: os/signal.signal_recv(0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: os/signal.loop() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by os/signal.init.0 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 53 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42043b980) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 54 [sleep]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: time.Sleep(0x3b9aca00) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc4203ee000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 203 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138c610, 0x72, 0xc421152ce8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc4212ee018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc4212ee018, 0xc420ffc000, 0x1000, 0x1000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Read(0xc4212ee000, 0xc420ffc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).Read(0xc4212ee000, 0xc420ffc000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*conn).Read(0xc42000e100, 0xc420ffc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).Read(0xc4210ca060, 0xc420f9c6e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: io.ReadAtLeast(0x1d59aa0, 0xc4210ca060, 0xc420f9c6e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: io.ReadFull(0x1d59aa0, 0xc4210ca060, 0xc420f9c6e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200cc2c0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200cc2c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 260 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421852840, 0xc422c8f700, 0x20, 0x20, 0x169d4e0, 0xc4206ab8a0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4228581b0, 0x1d718c0, 0xc421852840, 0x0, 0xc421852800) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc422c8f950, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4218520c0, 0xc420218280, 0xc422858090, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: reflect.Value.call(0xc4205c8840, 0xc420205228, 0x13, 0x1bda9a0, 0x4, 0xc422cc8d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: reflect.Value.Call(0xc4205c8840, 0xc420205228, 0x13, 0xc422cc8d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*service).call(0xc422784500, 0xc4214bc0f0, 0xc42291e118, 0x0, 0xc420764f80, 0xc42281c0a0, 0x19758e0, 0xc420218280, 0x16, 0x169e020, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).ServeRequest(0xc4214bc0f0, 0x1d72340, 0xc422784540, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d0e0, 0xc421014900, 0xc4214bc0f0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 41 [select, locked to thread]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: runtime.selectgo(0xc420505f50, 0xc4202ec960) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: runtime.ensureSigM.func1() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: runtime.goexit() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 55 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc4203f0300) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 56 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4203863c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 31 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4203fb580, 0x1d71840, 0xc42043b9c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 258 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4203ee0b0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 58 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc42021f860) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc420263740, 0xc42031ed90) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 88 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420e34340) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 47 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138cf00, 0x72, 0xc4200679a8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc420219c98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc420219c98, 0xc4205a2000, 0x1000, 0x1000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Read(0xc420219c80, 0xc4205a2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).Read(0xc420219c80, 0xc4205a2000, 0x1000, 0x1000, 0x458600, 0xc420092f00, 0x4) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*conn).Read(0xc420212860, 0xc4205a2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/http.(*persistConn).Read(0xc4203a8c60, 0xc4205a2000, 0x1000, 0x1000, 0xc420067b98, 0x405675, 0xc4202ecea0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).fill(0xc42057c240) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).Peek(0xc42057c240, 0x1, 0x0, 0x0, 0x1, 0xc420e78600, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/http.(*persistConn).readLoop(0xc4203a8c60) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by net/http.(*Transport).dialConn Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 202 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200cc2c0, 0xc421261d10, 0x30, 0xf9866a) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200cc2c0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc420de2740) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d320, 0xc42000e100, 0xc4205b8050) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d320, 0xc42000e100, 0xc4205b8050) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 111 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/http.(*persistConn).writeLoop(0xc420c2c7e0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by net/http.(*Transport).dialConn Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 97 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138c470, 0x72, 0xc422ccbce8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc420448018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc420448018, 0xc422a4e000, 0x1000, 0x1000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Read(0xc420448000, 0xc422a4e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).Read(0xc420448000, 0xc422a4e000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*conn).Read(0xc420204030, 0xc422a4e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).Read(0xc421adc0c0, 0xc422e69d00, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: io.ReadAtLeast(0x1d59aa0, 0xc421adc0c0, 0xc422e69d00, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: io.ReadFull(0x1d59aa0, 0xc421adc0c0, 0xc422e69d00, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4203ee0b0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4203ee0b0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:40 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 48 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/http.(*persistConn).writeLoop(0xc4203a8c60) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by net/http.(*Transport).dialConn Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 230 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422d6d860, 0xc422cfc240) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 215 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420f92000, 0xc4203e3d10, 0x30, 0xf9866a) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420f92000, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc420de2740) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d320, 0xc420212020, 0xc420603900) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d320, 0xc420212020, 0xc420603900) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 59 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4203a5630, 0x0, 0x0, 0x0, 0xc420087b00) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc42023a160) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:27:40 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 89 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc420012000, 0xc420d640c0, 0x1d6eb80, 0xc420e34340, 0xc420e622a0, 0xc420e624e0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420012000, 0xc420e35540) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 200 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc420012000, 0xc420d640c0, 0xc420e783c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420012000, 0xc420734ba0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 259 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4203ee0b0) Aug 23 15:27:40 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 119 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4203ee160, 0xc4203ded10, 0x30, 0xf9866a) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4203ee160, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc420de2740) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d320, 0xc420204610, 0xc420203860) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d320, 0xc420204610, 0xc420203860) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 139 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc420012000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc420012000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc420012000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420012000, 0xc420fa1460) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 140 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc420012000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420012000, 0xc420fa1470) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 141 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc420012000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420012000, 0xc420fa1480) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 142 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420ce6ba0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 143 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4212161a0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 144 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138cd60, 0x72, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc42058a018, 0x72, 0xc42020a000, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc42058a018, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Accept(0xc42058a000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).accept(0xc42058a000, 0xfb, 0xc420505758, 0x42bc79) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*TCPListener).accept(0xc420212078, 0xc420505770, 0xc4205057c8, 0xa17bd6) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*TCPListener).AcceptTCP(0xc420212078, 0x0, 0x435248, 0x1c66060) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc42099a3c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 145 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138cc90, 0x72, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc42058a098, 0x72, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc42058a098, 0xc421004000, 0x10000, 0x10000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).ReadFrom(0xc42058a080, 0xc421004000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).readFrom(0xc42058a080, 0xc421004000, 0x10000, 0x10000, 0xc421004000, 0x10000, 0xc420fe6e60, 0x4126b9, 0xc420057400) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*UDPConn).readFrom(0xc4202120c0, 0xc421004000, 0x10000, 0x10000, 0x101, 0xc420fe6e18, 0xc420fe6e90, 0x443cf7) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*UDPConn).ReadFrom(0xc4202120c0, 0xc421004000, 0x10000, 0x10000, 0x10000, 0x10000, 0x9ccc, 0x7d2b20c1, 0x2b96de0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc42099a3c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 146 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc42099a3c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 147 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc42099a3c0, 0x12a05f200, 0xc420cee1e0, 0xc420ed8360, 0xc420fa1990) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:110 +0xef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 148 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc42099a3c0, 0xc420ed8360) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 149 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc42099a3c0, 0x1dcd6500, 0xc420cee240, 0xc420ed8360, 0xc420fa19a0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 150 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc420fb6600) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 151 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc420fb6600) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 152 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420fb6600, 0x1bdd09a, 0x6, 0xc420ce7320) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 153 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420fb6600, 0x1bdba6c, 0x5, 0xc420ce7500) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 154 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420fb6600, 0x1bdbc6a, 0x5, 0xc420ce7710) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 188 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420452790, 0xc42125dd10, 0x30, 0xf9866a) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420452790, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc420de2740) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d320, 0xc42059e008, 0xc4205b9310) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d320, 0xc42059e008, 0xc4205b9310) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 156 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420cee2a0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420cee2a0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420cee2a0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 157 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420cee300, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420cee300, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420cee300) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 158 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420cee360, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420cee360, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420cee360) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 159 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420cee3c0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420cee3c0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420cee3c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 160 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138c880, 0x72, 0xc420fdd490) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc422394198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc422394198, 0xc420900000, 0x1000, 0x1000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Read(0xc422394180, 0xc420900000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).Read(0xc422394180, 0xc420900000, 0x1000, 0x1000, 0x1, 0xc420900000, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*conn).Read(0xc42000ec18, 0xc420900000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).Read(0xc42245d920, 0xc4223f500c, 0x1, 0x1, 0x1000, 0x1000, 0xc420fdd698) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).readTCP(0xc42099a3c0, 0x1d7d320, 0xc42000ec18, 0x2b96de0, 0x0, 0x0, 0x2b96de0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:912 +0xee Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc42099a3c0, 0xc422888240, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:772 +0x357 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc42099a3c0, 0xc422888240, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc42099a3c0, 0xc42094c600, 0x13, 0x20, 0x3, 0x3, 0xc420793780) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc420fb6600, 0xc42094c600, 0x13, 0x20, 0xc420793701, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc4200ee280, 0xc42094c600, 0x13, 0x20, 0xc420793780, 0x11, 0x10) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc420598740, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420d78450, 0xc4200ee280) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 161 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4200ee280) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 162 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4200ee280) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 163 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138ce30, 0x72, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc4205a9418, 0x72, 0xc420550700, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc4205a9418, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Accept(0xc4205a9400, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).accept(0xc4205a9400, 0x1, 0x1, 0xc4203e2f20) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*TCPListener).accept(0xc420204e60, 0x406b4a, 0xc420eb3d40, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*TCPListener).Accept(0xc420204e60, 0xc420eb3d40, 0x0, 0x3f800000, 0xc420de2740) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4200ee280, 0x1d71840, 0xc420de2740) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 164 [runnable]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4200d40a0, 0x3b9aca00, 0xc420078420) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 165 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4203a5400, 0x3b9aca00, 0xc420078420) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 166 [runnable]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc42052c120, 0x3b9aca00, 0xc420078420) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 167 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc42021f860, 0x3b9aca00, 0xc420078420) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 168 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4200ee280) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 169 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138cbc0, 0x72, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc420df6418, 0x72, 0xc420550200, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc420df6418, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Accept(0xc420df6400, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).accept(0xc420df6400, 0x402037, 0xc4200d5ca0, 0x1805ec0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*TCPListener).accept(0xc4202125d0, 0x458e40, 0xc420fe9e90, 0xc420fe9e98) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*TCPListener).Accept(0xc4202125d0, 0x1c65af8, 0xc4200d5c20, 0x1d71900, 0xc420d78f00) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/http.(*Server).Serve(0xc421216680, 0x1d6fc00, 0xc4202125d0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/http.Serve(0x1d6fc00, 0xc4202125d0, 0x1d5df00, 0xc420d789f0, 0x6461654ca9e66a0b, 0x4daba04c43417265) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420de2800, 0x1d6fc00, 0xc4202125d0, 0xc420fa1e30, 0xc420d786c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 170 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420df6480, 0xc420fa1e40, 0xc420ed8ba0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 171 [sleep]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: time.Sleep(0x246b40c5a) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc4203517a0, 0xc420df6480) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 198 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc420012000, 0xc420d64000, 0xc420ed8fc0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420012000, 0xc420734aa0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 110 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138ca20, 0x72, 0xc4200629a8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc420df6698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc420df6698, 0xc4208da000, 0x1000, 0x1000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Read(0xc420df6680, 0xc4208da000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).Read(0xc420df6680, 0xc4208da000, 0x1000, 0x1000, 0x458600, 0xc421236780, 0x4) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*conn).Read(0xc42000e618, 0xc4208da000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/http.(*persistConn).Read(0xc420c2c7e0, 0xc4208da000, 0x1000, 0x1000, 0xc420062b98, 0x405675, 0xc420e620c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).fill(0xc420788c60) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).Peek(0xc420788c60, 0x1, 0x0, 0x0, 0x1, 0xc420de5980, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/http.(*persistConn).readLoop(0xc420c2c7e0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by net/http.(*Transport).dialConn Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 189 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138c7b0, 0x72, 0xc420061ce8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc420dfa018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc420dfa018, 0xc4209a1000, 0x1000, 0x1000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Read(0xc420dfa000, 0xc4209a1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).Read(0xc420dfa000, 0xc4209a1000, 0x1000, 0x1000, 0xc420061dd8, 0x1, 0xc420061e08) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*conn).Read(0xc42059e008, 0xc4209a1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).Read(0xc42057d800, 0xc4210ae040, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: io.ReadAtLeast(0x1d59aa0, 0xc42057d800, 0xc4210ae040, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: io.ReadFull(0x1d59aa0, 0xc42057d800, 0xc4210ae040, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420452790, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420452790) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 120 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138caf0, 0x72, 0xc420febce8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc420e23098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc420e23098, 0xc420f15000, 0x1000, 0x1000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Read(0xc420e23080, 0xc420f15000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).Read(0xc420e23080, 0xc420f15000, 0x1000, 0x1000, 0x8e, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*conn).Read(0xc420204610, 0xc420f15000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).Read(0xc420d526c0, 0xc4206fce30, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: io.ReadAtLeast(0x1d59aa0, 0xc420d526c0, 0xc4206fce30, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: io.ReadFull(0x1d59aa0, 0xc420d526c0, 0xc4206fce30, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4203ee160, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4203ee160) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 121 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4203ee160) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 122 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4203ee160) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 216 [IO wait]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.runtime_pollWait(0x7fb7e138c540, 0x72, 0xc420fe8ce8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).wait(0xc420448418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*pollDesc).waitRead(0xc420448418, 0xc421efc000, 0x1000, 0x1000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: internal/poll.(*FD).Read(0xc420448400, 0xc421efc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*netFD).Read(0xc420448400, 0xc421efc000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net.(*conn).Read(0xc420212020, 0xc421efc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).Read(0xc4218539e0, 0xc4225f9d20, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: io.ReadAtLeast(0x1d59aa0, 0xc4218539e0, 0xc4225f9d20, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: io.ReadFull(0x1d59aa0, 0xc4218539e0, 0xc4225f9d20, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420f92000, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420f92000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 229 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422d6d860, 0xc422cfc1e0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 126 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc420012000, 0xc420d64000, 0x186080, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:171 +0x7a2 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc420012000, 0xc420d64000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:135 +0x258 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420012000, 0xc42075a300) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 191 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420452790) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 197 [runnable]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldBlock(0xc421eb5e40, 0x1a1c000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6607 +0x2a4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4200ee280, 0xc420206b01, 0xc420206be0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:300 +0x152 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4200ee280, 0xc420ed8e40, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4200ee280, 0xc420ed8e40) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420b1c070, 0xc4200ee280, 0xc420ed8e40) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 127 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc420012000, 0xc420d640c0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc420012000, 0xc420d640c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420012000, 0xc42075a320) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 190 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420452790) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 192 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422cfeea0, 0xc4213ff700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422d6d3e0, 0x1d718c0, 0xc422cfeea0, 0x0, 0xc422cfee00) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc4213ff950, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4203b0120, 0xc422394380, 0xc422d6d2c0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: reflect.Value.call(0xc4205c8840, 0xc42059e3d0, 0x13, 0x1bda9a0, 0x4, 0xc4213ffd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: reflect.Value.Call(0xc4205c8840, 0xc42059e3d0, 0x13, 0xc4213ffd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*service).call(0xc420d7a740, 0xc4205b93b0, 0xc422e69388, 0x0, 0xc420e9a900, 0xc420d518e0, 0x19758e0, 0xc422394380, 0x16, 0x169e020, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).ServeRequest(0xc4205b93b0, 0x1d72340, 0xc420d7a780, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d0e0, 0xc421014180, 0xc4205b93b0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 212 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421014000, 0xc420834000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).fill(0xc42134ac60) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).ReadByte(0xc42134ac60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4212e24e0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42149bc00) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42134ad80, 0x16e4b60, 0xc42149bc60) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42134ad80, 0x16e4b60, 0xc42149bc60, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421477c40, 0x16e4b60, 0xc42149bc60, 0xc42149bc60, 0xc420203978) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421477c40, 0xc42149bc60, 0x2d, 0xc422d12420) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).readRequestHeader(0xc420203950, 0x1d72340, 0xc421477c40, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200b7ea0, 0x8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).readRequest(0xc420203950, 0x1d72340, 0xc421477c40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).ServeRequest(0xc420203950, 0x1d72340, 0xc421477c40, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d0e0, 0xc421014000, 0xc420203950) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 204 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200cc2c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 205 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200cc2c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 206 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4200866c0, 0xc420fe1700, 0x20, 0x20, 0x169d4e0, 0xc4206ab8a0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421bf8060, 0x1d718c0, 0xc4200866c0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc420fe1950, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421258000, 0xc421265680, 0xc420f98c90, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: reflect.Value.call(0xc4205c8840, 0xc42000f2f8, 0x13, 0x1bda9a0, 0x4, 0xc420fe1d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: reflect.Value.Call(0xc4205c8840, 0xc42000f2f8, 0x13, 0xc420fe1d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*service).call(0xc4210d4400, 0xc4205b80f0, 0xc4209a64a8, 0x0, 0xc421265080, 0xc42128d900, 0x19758e0, 0xc421265680, 0x16, 0x169e020, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).ServeRequest(0xc4205b80f0, 0x1d72340, 0xc4210d4440, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d0e0, 0xc4204f8000, 0xc4205b80f0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 96 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4203ee0b0, 0xc422cc7d10, 0x30, 0xf9866a) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4203ee0b0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc420de2740) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d320, 0xc420204030, 0xc4214bc050) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d320, 0xc420204030, 0xc4214bc050) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 217 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420f92000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 246 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421014300, 0xc422251000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).fill(0xc422db1860) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).ReadByte(0xc422db1860, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422d043c0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422e2edc0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422db18c0, 0x16e4b60, 0xc422e2ee40) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422db18c0, 0x16e4b60, 0xc422e2ee40, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4226dcc40, 0x16e4b60, 0xc422e2ee40, 0xc422e2ee40, 0xc4205b93d8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4226dcc40, 0xc422e2ee40, 0x2d, 0xc422fb0e40) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).readRequestHeader(0xc4205b93b0, 0x1d72340, 0xc4226dcc40, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200b7ea0, 0x8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).readRequest(0xc4205b93b0, 0x1d72340, 0xc4226dcc40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).ServeRequest(0xc4205b93b0, 0x1d72340, 0xc4226dcc40, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d0e0, 0xc421014300, 0xc4205b93b0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 218 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420f92000) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 219 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422cfefc0, 0xc421149700, 0x20, 0x20, 0x169d4e0, 0xc4206ab8a0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422d6d680, 0x1d718c0, 0xc422cfefc0, 0x0, 0xc422cfef00) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc421149950, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421add9e0, 0xc422394480, 0xc422d6d590, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: reflect.Value.call(0xc4205c8840, 0xc420212758, 0x13, 0x1bda9a0, 0x4, 0xc420063d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: reflect.Value.Call(0xc4205c8840, 0xc420212758, 0x13, 0xc420063d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*service).call(0xc421920d80, 0xc4206039f0, 0xc422e69978, 0x0, 0xc4225aa680, 0xc4203b4a60, 0x19758e0, 0xc422394480, 0x16, 0x169e020, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).ServeRequest(0xc4206039f0, 0x1d72340, 0xc421920dc0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d0e0, 0xc4210140c0, 0xc4206039f0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 247 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421014480, 0xc4226a8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).fill(0xc422db1c20) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).ReadByte(0xc422db1c20, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422d04990, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422e2efc0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422db1c80, 0x16e4b60, 0xc422e2f020) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422db1c80, 0x16e4b60, 0xc422e2f020, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4226dce40, 0x16e4b60, 0xc422e2f020, 0xc422e2f020, 0xc420603a18) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4226dce40, 0xc422e2f020, 0x2d, 0xc422fb1020) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).readRequestHeader(0xc4206039f0, 0x1d72340, 0xc4226dce40, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200b7ea0, 0x8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).readRequest(0xc4206039f0, 0x1d72340, 0xc4226dce40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).ServeRequest(0xc4206039f0, 0x1d72340, 0xc4226dce40, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d0e0, 0xc421014480, 0xc4206039f0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 248 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204f80c0, 0xc4215c4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).fill(0xc421860060) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: bufio.(*Reader).ReadByte(0xc421860060, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422d04f30, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422e2f1e0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4218600c0, 0x16e4b60, 0xc422e2f240) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4218600c0, 0x16e4b60, 0xc422e2f240, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4226dd300, 0x16e4b60, 0xc422e2f240, 0xc422e2f240, 0xc4214bc118) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4226dd300, 0xc422e2f240, 0x2d, 0xc422fb1200) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).readRequestHeader(0xc4214bc0f0, 0x1d72340, 0xc4226dd300, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200b7ea0, 0x8) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).readRequest(0xc4214bc0f0, 0x1d72340, 0xc4226dd300, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: net/rpc.(*Server).ServeRequest(0xc4214bc0f0, 0x1d72340, 0xc4226dd300, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc420de2740, 0x1d7d0e0, 0xc4204f80c0, 0xc4214bc0f0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 220 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4206d8be0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 221 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4206d8be0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 222 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4203a5400, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4200ee280) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 224 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc42052c120) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 225 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc42052c120) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 274 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc4224c6240) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 275 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4225e2a00, 0xc421262ce8, 0x20, 0x20, 0x18, 0xc422e2f5a0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422d056e0, 0x1d71840, 0xc4225e2a00, 0xc422e2f5a0, 0x184987) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420206be0, 0xc422c8ded0, 0x184987, 0x1d71840, 0xc4225e2a00, 0xc42058a360, 0xc4224c6210, 0xc4207b0b40, 0x24, 0x2bba400) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc42058a300, 0x1d71840, 0xc4225e2a00, 0x184987, 0xc4226dd540, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc42058a300, 0x1d71840, 0xc4225e2a00) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 276 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4225e2b00, 0xc420fe3770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422d6d920, 0x1d71840, 0xc4225e2b00, 0x0, 0x1849b5) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420206be0, 0xc420fe3960, 0x1849b5, 0x1d71840, 0xc4225e2b00, 0x552bde, 0xc420fe3950, 0xc420fe3970, 0xf33ba5, 0xc42014c598) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc42014c540, 0x1d71840, 0xc4225e2b00, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc42014c540) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 277 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4225e2a40, 0xc422ac5a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224c6ff0, 0x1d71840, 0xc4225e2a40, 0xc4224c7110, 0x1833a4) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420206be0, 0xc422ac5c68, 0x1833a4, 0x1d71840, 0xc4225e2a40, 0x552bde, 0xc4203a5180, 0x2, 0xc420d5a500, 0x40) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc4225e2b40, 0x1833a4, 0x3b, 0xc422cc6de8, 0x1, 0x1) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc4225e2b40) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 278 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc4225e2bc0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 279 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc42099a5a0, 0x1d71840, 0xc4225e2a40) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 249 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc421a143c0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: goroutine 280 [select]: Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4226dd600, 0xc422c5fd28, 0x20, 0x20, 0xc420468c00, 0x1bd9a98) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224c6fc0, 0x1d71840, 0xc4226dd600, 0xc422d3f340, 0x1849b5) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420206be0, 0xc422ac1f20, 0x1849b5, 0x1d71840, 0xc4226dd600, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc421a143c0, 0x1849b5, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc421a143c0, 0x1849b5, 0xc42056a480) Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:27:40 ip-172-24-20-236 nomad.sh[25753]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:27:40 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:27:40 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:27:40 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: ==> Starting Nomad agent... Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: ==> Nomad agent configuration: Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: Client: false Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: Log Level: INFO Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: Region: amrv01 (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: Server: true Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: Version: 0.8.4 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.216630 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: Attempting re-join to previously known node: consulnomad-01.aaf01: 172.26.1.69:4648 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.217147 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397344 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397368 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397380 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397392 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397403 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397700 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397722 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397733 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397958 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397972 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397983 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.397993 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.398003 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.398014 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.398024 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.398035 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.398046 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [WARN] memberlist: Refuting an alive message Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41 [INFO] serf: Re-joined to previously known node: consulnomad-01.aaf01: 172.26.1.69:4648 Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.398276 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:41 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:41.398289 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:43 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:43 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:45 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:45 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 37 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46.089873 [INFO] nomad: cluster leadership acquired Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: 2018/08/23 15:27:46 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: fatal error: concurrent map read and map write Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 282 [running]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc42101dd20 sp=0xc42101dd00 pc=0x42d161 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: runtime.mapaccess2(0x1844be0, 0xc4201b3140, 0xc42101dde0, 0xc42000f560, 0x8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc42101dd68 sp=0xc42101dd20 pc=0x409c75 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc420392000, 0x1d6dd40, 0xc42000f560, 0x0, 0xed3109f42) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:156 +0xdc fp=0xc42101de50 sp=0xc42101dd68 pc=0xf215ac Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc42033abe0, 0x1d71840, 0xc422c44900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc42101dfc8 sp=0xc42101de50 pc=0xf56bb7 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: runtime.goexit() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc42101dfd0 sp=0xc42101dfc8 pc=0x45c711 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 1 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc4201317a0, 0xc4202b3400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc4201317a0, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4203eb400, 0xc4203eb400, 0xc42049b500, 0x37) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204d7f68) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: main.main() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 5 [syscall]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: os/signal.signal_recv(0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: os/signal.loop() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by os/signal.init.0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 280 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4206bfc20) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 138 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4204533f0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 103 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42016a2c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 53 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200b62c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b62c0, 0xc4201fa6b0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 231 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b62c0, 0xc4204a3b00, 0xc422dca360) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b62c0, 0xc422d3bb00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 39 [select, locked to thread]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: runtime.selectgo(0xc420050f50, 0xc420090720) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: runtime.ensureSigM.func1() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: runtime.goexit() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 40 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42042fe80) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 41 [sleep]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: time.Sleep(0x3b9aca00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420453970) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 42 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc4204a2540) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 43 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc42044cf00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 187 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422733ce0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 45 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4201b4a90) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0xc42017f3d0, 0xc42017f3e0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4201b3590, 0xc42020fe20) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 182 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc42042ff40, 0x1d71840, 0xc4202b8800) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 170 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42016a2c0, 0xc420e25d10, 0x30, 0xf9866a) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42016a2c0, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4205e1400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc420202368, 0xc4206fc0a0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc420202368, 0xc4206fc0a0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 71 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4200a91d0, 0x1c636a0, 0xc4205ac000, 0x1d71840, 0xc420593240) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4205ac000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 139 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4204533f0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 140 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42016c600, 0xc421357000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).fill(0xc42031d2c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).ReadByte(0xc42031d2c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4207ea240, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420a86c40) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42031d320, 0x16e4b60, 0xc420a86ca0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42031d320, 0x16e4b60, 0xc420a86ca0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421149c00, 0x16e4b60, 0xc420a86ca0, 0xc420a86ca0, 0xc4204da208) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421149c00, 0xc420a86ca0, 0x2d, 0xc420c866c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).readRequestHeader(0xc4204da1e0, 0x1d72340, 0xc421149c00, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42043c870, 0x8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).readRequest(0xc4204da1e0, 0x1d72340, 0xc421149c00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).ServeRequest(0xc4204da1e0, 0x1d72340, 0xc421149c00, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d0e0, 0xc42016c600, 0xc4204da1e0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 104 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42016c540, 0xc420fbe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).fill(0xc4204d8d80) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).ReadByte(0xc4204d8d80, 0x101000000000016, 0x0, 0x2d) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42058c360, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420cdca40) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4204d8de0, 0x16e4b60, 0xc420cdcaa0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4204d8de0, 0x16e4b60, 0xc420cdcaa0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4209b8a80, 0x16e4b60, 0xc420cdcaa0, 0xc420cdcaa0, 0xc420cd2488) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4209b8a80, 0xc420cdcaa0, 0x2d, 0xc420c864c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).readRequestHeader(0xc420cd2460, 0x1d72340, 0xc4209b8a80, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).readRequest(0xc420cd2460, 0x1d72340, 0xc4209b8a80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).ServeRequest(0xc420cd2460, 0x1d72340, 0xc4209b8a80, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d0e0, 0xc42016c540, 0xc420cd2460) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 101 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8b950, 0x72, 0xc421024ce8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc420b7c698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc420b7c698, 0xc420a71000, 0x1000, 0x1000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc420b7c680, 0xc420a71000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc420b7c680, 0xc420a71000, 0x1000, 0x1000, 0xc4204b3dd8, 0x1, 0xc4204b3e08) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc420202368, 0xc420a71000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).Read(0xc42082c600, 0xc420b6f7a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadAtLeast(0x1d59aa0, 0xc42082c600, 0xc420b6f7a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadFull(0x1d59aa0, 0xc42082c600, 0xc420b6f7a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42016a2c0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42016a2c0) Aug 23 15:27:46 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 284 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420470b40) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 69 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8bf00, 0x72, 0xc4200669a8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc4203a4298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc4203a4298, 0xc420572000, 0x1000, 0x1000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc4203a4280, 0xc420572000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc4203a4280, 0xc420572000, 0x1000, 0x1000, 0x458600, 0xc42048d500, 0x4) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc4202020e8, 0xc420572000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*persistConn).Read(0xc4204065a0, 0xc420572000, 0x1000, 0x1000, 0xc420066b98, 0x405675, 0xc420302540) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).fill(0xc4200869c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).Peek(0xc4200869c0, 0x1, 0x0, 0x0, 0x1, 0xc420e16a20, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*persistConn).readLoop(0xc4204065a0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by net/http.(*Transport).dialConn Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 283 [select]: Aug 23 15:27:46 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420470b40) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 70 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*persistConn).writeLoop(0xc4204065a0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by net/http.(*Transport).dialConn Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 102 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42016a2c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 52 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200b62c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200b62c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200b62c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b62c0, 0xc4201fa6a0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:46 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 239 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc420758000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 54 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200b62c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b62c0, 0xc4201fa6d0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 55 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420f49500) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 56 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420476410) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 57 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8bd60, 0x72, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc42039b498, 0x72, 0xc4202b8000, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc42039b498, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Accept(0xc42039b480, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).accept(0xc42039b480, 0xc42000e4f8, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*TCPListener).accept(0xc420202168, 0x458e40, 0xc420387f90, 0xc420387f98) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*TCPListener).AcceptTCP(0xc420202168, 0x1c63658, 0xc420fc6690, 0xc42000e4f8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420fc6690) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 58 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8bc90, 0x72, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc42039b518, 0x72, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc42039b518, 0xc42323a000, 0x10000, 0x10000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).ReadFrom(0xc42039b500, 0xc42323a000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).readFrom(0xc42039b500, 0xc42323a000, 0x10000, 0x10000, 0xe9, 0xe9, 0x1d62b40, 0x412539, 0xc422ba1e40) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*UDPConn).readFrom(0xc420202188, 0xc42323a000, 0x10000, 0x10000, 0x101, 0xc422ba1e18, 0xc422ba1e90, 0x443cf7) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*UDPConn).ReadFrom(0xc420202188, 0xc42323a000, 0x10000, 0x10000, 0x10000, 0x10000, 0x7151, 0x142b91f0c, 0x2b96de0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420fc6690) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 59 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420fc6690) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 60 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420fc6690, 0x12a05f200, 0xc421300780, 0xc420fbb5c0, 0xc4201fadf0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 61 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420fc6690, 0xc420fbb5c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 62 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420fc6690, 0x1dcd6500, 0xc4213007e0, 0xc420fbb5c0, 0xc4201fae00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 63 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4201e9680) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 64 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4201e9680) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 65 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4201e9680, 0x1bdd09a, 0x6, 0xc420f49650) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 146 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4201e9680, 0x1bdba6c, 0x5, 0xc420f49680) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 147 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4201e9680, 0x1bdbc6a, 0x5, 0xc420f496b0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 105 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4204533f0, 0xc421142d10, 0x30, 0xf9866a) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4204533f0, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4205e1400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc420216c40, 0xc420cd2ff0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc420216c40, 0xc420cd2ff0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 149 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421300840, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421300840, 0x1dcd6500, 0x1bd9928, 0x2, 0x1bdab30, 0x4, 0xc42136a000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421300840) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 150 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4213008a0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4213008a0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4213008a0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 151 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421300900, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421300900, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421300900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 152 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421300960, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421300960, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421300960) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 153 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8b7b0, 0x77, 0xc420545b00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc422153498, 0x77, 0xc420527000, 0xc420527028, 0x436544) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitWrite(0xc422153498, 0xc420527000, 0x28, 0x611b26) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).WaitWrite(0xc422153480, 0x1c65988, 0x1d718c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:440 +0x37 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).connect(0xc422153480, 0x1d718c0, 0xc421520e40, 0x0, 0x0, 0x1d5c800, 0xc421fb6b20, 0x0, 0x0, 0x0, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:152 +0x299 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).dial(0xc422153480, 0x1d718c0, 0xc421520e40, 0x1d7a200, 0x0, 0x1d7a200, 0xc4228e2c60, 0xc420527248, 0x62f2be) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/sock_posix.go:142 +0xe9 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.socket(0x1d718c0, 0xc421520e40, 0x1bda392, 0x3, 0x2, 0x1, 0x0, 0x0, 0x1d7a200, 0x0, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/sock_posix.go:93 +0x1a6 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.internetSocket(0x1d718c0, 0xc421520e40, 0x1bda392, 0x3, 0x1d7a200, 0x0, 0x1d7a200, 0xc4228e2c60, 0x1, 0x0, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/ipsock_posix.go:141 +0x129 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.doDialTCP(0x1d718c0, 0xc421520e40, 0x1bda392, 0x3, 0x0, 0xc4228e2c60, 0x2bb8aa0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/tcpsock_posix.go:62 +0xb9 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.dialTCP(0x1d718c0, 0xc421520e40, 0x1bda392, 0x3, 0x0, 0xc4228e2c60, 0x83a7ccd51, 0x2b96de0, 0x6fc238266) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/tcpsock_posix.go:58 +0xe4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.dialSingle(0x1d718c0, 0xc421520e40, 0xc422153400, 0x1d62b00, 0xc4228e2c60, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/dial.go:547 +0x375 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.dialSerial(0x1d718c0, 0xc421520e40, 0xc422153400, 0xc422e65380, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/dial.go:515 +0x22d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*Dialer).DialContext(0xc420527848, 0x1d71880, 0xc42003e028, 0x1bda392, 0x3, 0xc421f61630, 0x10, 0x0, 0x0, 0x0, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/dial.go:397 +0x678 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*Dialer).Dial(0xc420527848, 0x1bda392, 0x3, 0xc421f61630, 0x10, 0x20, 0x20, 0x17503e0, 0x2ade3c8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/dial.go:320 +0x75 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc420fc6690, 0xc421f61550, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:758 +0x14e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc420fc6690, 0xc421f61550, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc420fc6690, 0xc42064cc00, 0x13, 0x20, 0x3, 0x3, 0xc420ccb360) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc4201e9680, 0xc42064cc00, 0x13, 0x20, 0xc420ccb301, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc4200e1900, 0xc42064cc00, 0x13, 0x20, 0xc420ccb360, 0x10, 0x10) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc4204b4f40, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc4213ab6b0, 0xc4200e1900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 154 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4200e1900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 155 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4200e1900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 156 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8be30, 0x72, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc4203a5898, 0x72, 0xc421148100, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc4203a5898, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Accept(0xc4203a5880, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).accept(0xc4203a5880, 0x1, 0x1, 0xc420065f20) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*TCPListener).accept(0xc420202698, 0x406b4a, 0xc420eb8660, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*TCPListener).Accept(0xc420202698, 0xc420eb8660, 0x0, 0x3f800000, 0xc4205e1400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4200e1900, 0x1d71840, 0xc4205e1400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 157 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc42033abe0, 0x3b9aca00, 0xc420090ba0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 158 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc42043cbe0, 0x3b9aca00, 0xc420090ba0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 159 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420470b40, 0x3b9aca00, 0xc420090ba0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 160 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4201b4a90, 0x3b9aca00, 0xc420090ba0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 161 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4200e1900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 162 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8bbc0, 0x72, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc420b89198, 0x72, 0xc421148400, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc420b89198, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Accept(0xc420b89180, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).accept(0xc420b89180, 0x402037, 0xc4210aac60, 0x1805ec0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*TCPListener).accept(0xc4202027e0, 0x458e40, 0xc421029e90, 0xc421029e98) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*TCPListener).Accept(0xc4202027e0, 0x1c65af8, 0xc4210aabe0, 0x1d71900, 0xc42114c420) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*Server).Serve(0xc420f1d380, 0x1d6fc00, 0xc4202027e0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.Serve(0x1d6fc00, 0xc4202027e0, 0x1d5df00, 0xc42114c360, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc4205e14c0, 0x1d6fc00, 0xc4202027e0, 0xc4201fb350, 0xc4213ab770) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 163 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420b89200, 0xc4201fb370, 0xc420f587e0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 164 [sleep]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: time.Sleep(0x229f36e67) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc4201317a0, 0xc420b89200) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 172 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200c02c0, 0xc421018d10, 0x30, 0xf9866a) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200c02c0, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4205e1400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc4202024f0, 0xc4206fc320) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc4202024f0, 0xc4206fc320) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 166 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8ba20, 0x72, 0xc42101a9a8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc420b7c418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc420b7c418, 0xc421358000, 0x1000, 0x1000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc420b7c400, 0xc421358000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc420b7c400, 0xc421358000, 0x1000, 0x1000, 0x458600, 0xc420545b00, 0x4) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc420202088, 0xc421358000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*persistConn).Read(0xc420ee66c0, 0xc421358000, 0x1000, 0x1000, 0xc42101ab98, 0x405675, 0xc420e6cde0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).fill(0xc4211b81e0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).Peek(0xc4211b81e0, 0x1, 0x0, 0x0, 0x1, 0xc420d8b1a0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*persistConn).readLoop(0xc420ee66c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by net/http.(*Transport).dialConn Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 167 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*persistConn).writeLoop(0xc420ee66c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by net/http.(*Transport).dialConn Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 285 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc4228e37a0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 137 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8baf0, 0x72, 0xc420e26ce8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc420e13998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc420e13998, 0xc421356000, 0x1000, 0x1000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc420e13980, 0xc421356000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc420e13980, 0xc421356000, 0x1000, 0x1000, 0x90, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc420216c40, 0xc421356000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).Read(0xc4201f57a0, 0xc420b6f8a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadAtLeast(0x1d59aa0, 0xc4201f57a0, 0xc420b6f8a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadFull(0x1d59aa0, 0xc4201f57a0, 0xc420b6f8a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4204533f0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4204533f0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 173 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8b880, 0x72, 0xc421144ce8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc420b7c818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc420b7c818, 0xc4208f1000, 0x1000, 0x1000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc420b7c800, 0xc4208f1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc420b7c800, 0xc4208f1000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc4202024f0, 0xc4208f1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).Read(0xc4211b9020, 0xc42078c770, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadAtLeast(0x1d59aa0, 0xc4211b9020, 0xc42078c770, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadFull(0x1d59aa0, 0xc4211b9020, 0xc42078c770, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200c02c0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200c02c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 174 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200c02c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 175 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200c02c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 176 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421256ba0, 0xc420e5d700, 0x20, 0x20, 0x169d4e0, 0xc4202d0ea0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420dbb860, 0x1d718c0, 0xc421256ba0, 0x0, 0xc421256b00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1900, 0xc420e5d950, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420b40ea0, 0xc4213e4000, 0xc420dbb710, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: reflect.Value.call(0xc420087800, 0xc4202029e8, 0x13, 0x1bda9a0, 0x4, 0xc420e5dd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: reflect.Value.Call(0xc420087800, 0xc4202029e8, 0x13, 0xc420e5dd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*service).call(0xc4205e0940, 0xc4206fc3c0, 0xc42073d800, 0x0, 0xc42081fd00, 0xc4211ba9a0, 0x19758e0, 0xc4213e4000, 0x16, 0x169e020, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).ServeRequest(0xc4206fc3c0, 0x1d72340, 0xc4205e09c0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d0e0, 0xc420bde6c0, 0xc4206fc3c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 277 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).connect.func2(0x1d718c0, 0xc421520e40, 0xc422153480, 0xc422da32c0, 0xc422da3260) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:129 +0xde Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by net.(*netFD).connect Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:128 +0x286 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 279 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4206bfc20) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 224 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422922000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 190 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b62c0, 0xc4204a3a40, 0xc422da2ae0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b62c0, 0xc422db1e60) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 281 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc42043cbe0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4200e1900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 107 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42016a160, 0xc42101cd10, 0x30, 0xf9866a) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42016a160, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4205e1400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc4204a41c8, 0xc4210fec80) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc4204a41c8, 0xc4210fec80) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 108 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8b470, 0x72, 0xc4211bdce8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc420b88798, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc420b88798, 0xc420ab8000, 0x1000, 0x1000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc420b88780, 0xc420ab8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc420b88780, 0xc420ab8000, 0x1000, 0x1000, 0xc4211bddd8, 0x1, 0xc4211bde08) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc4204a41c8, 0xc420ab8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).Read(0xc421179d40, 0xc421450b00, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadAtLeast(0x1d59aa0, 0xc421179d40, 0xc421450b00, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadFull(0x1d59aa0, 0xc421179d40, 0xc421450b00, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42016a160, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42016a160) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 109 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42016a160) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 110 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42016a160) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 111 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4204d9800, 0xc4211f5700, 0x20, 0x20, 0x169d4e0, 0xc42035b100) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4218c4ea0, 0x1d718c0, 0xc4204d9800, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1900, 0xc4211f5950, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42031d3e0, 0xc420810a80, 0xc420c03290, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: reflect.Value.call(0xc420087800, 0xc4204a4fa8, 0x13, 0x1bda9a0, 0x4, 0xc421019d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: reflect.Value.Call(0xc420087800, 0xc4204a4fa8, 0x13, 0xc421019d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*service).call(0xc42121b740, 0xc4210fed20, 0xc420e0dde8, 0x0, 0xc420796e00, 0xc4204d54c0, 0x19758e0, 0xc420810a80, 0x16, 0x169e020, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).ServeRequest(0xc4210fed20, 0x1d72340, 0xc42121b780, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d0e0, 0xc4204a2000, 0xc4210fed20) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 186 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8b610, 0x72, 0xc422cd1ce8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc420a80118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc420a80118, 0xc422c47000, 0x1000, 0x1000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc420a80100, 0xc422c47000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc420a80100, 0xc422c47000, 0x1000, 0x1000, 0xc422cd1dd8, 0x0, 0xc422cd1e08) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc420216cb0, 0xc422c47000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).Read(0xc42194dda0, 0xc422cca480, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadAtLeast(0x1d59aa0, 0xc42194dda0, 0xc422cca480, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadFull(0x1d59aa0, 0xc42194dda0, 0xc422cca480, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422733ce0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422733ce0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 225 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422922000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 197 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205ac0b0, 0xc420060d10, 0x30, 0xf9866a) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205ac0b0, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4205e1400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc4202020c8, 0xc420cd22d0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc4202020c8, 0xc420cd22d0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 198 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8b3a0, 0x72, 0xc422cd2ce8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc42241a018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc42241a018, 0xc422d60000, 0x1000, 0x1000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc42241a000, 0xc422d60000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc42241a000, 0xc422d60000, 0x1000, 0x1000, 0xc422cd2dd8, 0x0, 0xc422cd2e08) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc4202020c8, 0xc422d60000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).Read(0xc421fe64e0, 0xc422c3ff20, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadAtLeast(0x1d59aa0, 0xc421fe64e0, 0xc422c3ff20, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadFull(0x1d59aa0, 0xc421fe64e0, 0xc422c3ff20, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205ac0b0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205ac0b0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 199 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205ac0b0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 200 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205ac0b0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 188 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422733ce0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 223 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8b200, 0x72, 0xc421022ce8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc420a81498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc420a81498, 0xc421cf1000, 0x1000, 0x1000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc420a81480, 0xc421cf1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc420a81480, 0xc421cf1000, 0x1000, 0x1000, 0xc421022dd8, 0x1, 0xc421022e08) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc420217308, 0xc421cf1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).Read(0xc421b7a9c0, 0xc42285fe80, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadAtLeast(0x1d59aa0, 0xc421b7a9c0, 0xc42285fe80, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: io.ReadFull(0x1d59aa0, 0xc421b7a9c0, 0xc42285fe80, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422922000, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422922000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 204 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc422bec480, 0xc421cf5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).fill(0xc422dbeae0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).ReadByte(0xc422dbeae0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422dceff0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422d72c00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422dbeb40, 0x16e4b60, 0xc422d72c60) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422dbeb40, 0x16e4b60, 0xc422d72c60, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422d84b80, 0x16e4b60, 0xc422d72c60, 0xc422d72c60, 0xc4210fed48) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422d84b80, 0xc422d72c60, 0x2d, 0xc4224e38a0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).readRequestHeader(0xc4210fed20, 0x1d72340, 0xc422d84b80, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42043c870, 0x8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).readRequest(0xc4210fed20, 0x1d72340, 0xc422d84b80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).ServeRequest(0xc4210fed20, 0x1d72340, 0xc422d84b80, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d0e0, 0xc422bec480, 0xc4210fed20) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 220 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4200b62c0, 0xc4204a3a40, 0x186084, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:171 +0x7a2 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b62c0, 0xc4204a3a40) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:135 +0x258 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b62c0, 0xc422ca75a0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 221 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b62c0, 0xc4204a3b00, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b62c0, 0xc4204a3b00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b62c0, 0xc422ca75c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 222 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422922000, 0xc422cd0d10, 0x30, 0xf9866a) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422922000, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4205e1400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc420217308, 0xc42062e8c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d320, 0xc420217308, 0xc42062e8c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 209 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422861bc0, 0xc4210312c8, 0x20, 0x20, 0x5, 0x1bd9a98) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4229ec420, 0x1d718c0, 0xc422861bc0, 0x0, 0xc422861b00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1900, 0xc421031500, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Node).List(0xc4200875c0, 0xc422861aa0, 0xc422cdff80, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:1198 +0x27c Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: reflect.Value.call(0xc4200878c0, 0xc4202025e8, 0x13, 0x1bda9a0, 0x4, 0xc4210318f8, 0x3, 0x3, 0x192ba60, 0x19, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: reflect.Value.Call(0xc4200878c0, 0xc4202025e8, 0x13, 0xc4211c18f8, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*service).call(0xc4202b8c00, 0xc42043cc30, 0xc42285fd50, 0x0, 0xc4203a5580, 0xc422ce7980, 0x1975760, 0xc422861aa0, 0x16, 0x169e120, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).ServeRequest(0xc42043cc30, 0x1d71b80, 0xc422d59d00, 0xc420542400, 0xa) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc4200e1900, 0x1be205d, 0x9, 0x1975760, 0xc422861a40, 0x169e120, 0xc422cdff50, 0xc422861a01, 0xc422cdff50) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc4202adf10, 0x1be205d, 0x9, 0x1975760, 0xc422861a40, 0x169e120, 0xc422cdff50, 0xc4203ef490, 0x1e) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest(0xc4205e14c0, 0x1d6df80, 0xc4228619e0, 0xc4201dca00, 0xc4203ef4c0, 0xc4211c1b58, 0x38, 0xc4211c1b50) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/node_endpoint.go:24 +0x1b3 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest-fm(0x1d6df80, 0xc4228619e0, 0xc4201dca00, 0xc4201dca00, 0xc422902dc0, 0x1e, 0xbed7c7f08513f161) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:145 +0x48 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc4228619e0, 0xc4201dca00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.HandlerFunc.ServeHTTP(0xc421014860, 0x1d6df80, 0xc4228619e0, 0xc4201dca00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*ServeMux).ServeHTTP(0xc4213ab770, 0x1d6df80, 0xc4228619e0, 0xc4201dca00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc421cc0460, 0xc4201dca00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.HandlerFunc.ServeHTTP(0xc42114c360, 0x1d6fec0, 0xc421cc0460, 0xc4201dca00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.serverHandler.ServeHTTP(0xc420f1d380, 0x1d6fec0, 0xc421cc0460, 0xc4201dca00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*conn).serve(0xc4210aa960, 0x1d71840, 0xc422d59b80) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by net/http.(*Server).Serve Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 242 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422861680, 0xc422b8b208, 0x20, 0x20, 0xc422bf8f00, 0xc422cdf260) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422cdf6e0, 0x1d718c0, 0xc422861680, 0x0, 0xc422861600) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1900, 0xc422b8b440, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Job).List(0xc4204a45e8, 0xc422860900, 0xc422cdf260, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/job_endpoint.go:925 +0x2ea Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: reflect.Value.call(0xc42051a060, 0xc4204a4ad0, 0x13, 0x1bda9a0, 0x4, 0xc421031898, 0x3, 0x3, 0x192b7e0, 0x19, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: reflect.Value.Call(0xc42051a060, 0xc4204a4ad0, 0x13, 0xc4211c1898, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*service).call(0xc4203960c0, 0xc42043cc30, 0xc42285f630, 0x0, 0xc42039ae80, 0xc422ce6e20, 0x1974e60, 0xc422860900, 0x16, 0x169dd20, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).ServeRequest(0xc42043cc30, 0x1d71b80, 0xc422dafb80, 0xc420542400, 0xa) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc4200e1900, 0x1be08be, 0x8, 0x1974e60, 0xc4228608a0, 0x169dd20, 0xc422cdf200, 0xc422860801, 0xc422cdf200) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc4202adf10, 0x1be08be, 0x8, 0x1974e60, 0xc4228608a0, 0x169dd20, 0xc422cdf200, 0x0, 0xc420542400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).jobListRequest(0xc4205e14c0, 0x1d6df80, 0xc422860840, 0xc4201dc900, 0x44774f, 0xc421da9200, 0xc4203eecb0, 0x1d) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:33 +0xe8 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest(0xc4205e14c0, 0x1d6df80, 0xc422860840, 0xc4201dc900, 0xc4203eece0, 0xc4211c1b58, 0x38, 0xc4211c1b50) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:18 +0x1a6 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest-fm(0x1d6df80, 0xc422860840, 0xc4201dc900, 0xc4201dc900, 0xc422902880, 0x1d, 0xbed7c7f0850e0d71) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:141 +0x48 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc422860840, 0xc4201dc900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.HandlerFunc.ServeHTTP(0xc421014800, 0x1d6df80, 0xc422860840, 0xc4201dc900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*ServeMux).ServeHTTP(0xc4213ab770, 0x1d6df80, 0xc422860840, 0xc4201dc900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc421cc0380, 0xc4201dc900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.HandlerFunc.ServeHTTP(0xc42114c360, 0x1d6fec0, 0xc421cc0380, 0xc4201dc900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.serverHandler.ServeHTTP(0xc420f1d380, 0x1d6fec0, 0xc421cc0380, 0xc4201dc900) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*conn).serve(0xc4210aabe0, 0x1d71840, 0xc422dafa00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by net/http.(*Server).Serve Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 243 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8b2d0, 0x72, 0xc420387658) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc4222e1898, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc4222e1898, 0xc422cdef00, 0x1, 0x1) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc4222e1880, 0xc422cdefa1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc4222e1880, 0xc422cdefa1, 0x1, 0x1, 0xc4203877a0, 0x0, 0xc420e6c4e0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc420202e88, 0xc422cdefa1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*connReader).backgroundRead(0xc422cdef90) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 244 [IO wait]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.runtime_pollWait(0x7f976ec8b6e0, 0x72, 0xc420388658) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).wait(0xc4222e1818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*pollDesc).waitRead(0xc4222e1818, 0xc422cdfd00, 0x1, 0x1) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: internal/poll.(*FD).Read(0xc4222e1800, 0xc422cdfd21, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*netFD).Read(0xc4222e1800, 0xc422cdfd21, 0x1, 0x1, 0xc, 0xc, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net.(*conn).Read(0xc420202e80, 0xc422cdfd21, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/http.(*connReader).backgroundRead(0xc422cdfd10) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 245 [runnable]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldBlock(0xc4212094a0, 0x1a1c000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6607 +0x2a4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4200e1900, 0xc4202d0b01, 0xc4202d0b40) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:300 +0x152 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4200e1900, 0xc422ddcd80, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4200e1900, 0xc422ddcd80) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420c34200, 0xc4200e1900, 0xc422ddcd80) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 278 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421e34240, 0xc4223bc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).fill(0xc421521200) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: bufio.(*Reader).ReadByte(0xc421521200, 0x100000000000016, 0x0, 0x2d) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4228e2c90, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4228ba580) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421521260, 0x16e4b60, 0xc4228ba600) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421521260, 0x16e4b60, 0xc4228ba600, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422c444c0, 0x16e4b60, 0xc4228ba600, 0xc4228ba600, 0xc42062e988) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422c444c0, 0xc4228ba600, 0x2d, 0xc421fb6ca0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).readRequestHeader(0xc42062e960, 0x1d72340, 0xc422c444c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42043c870, 0x8) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).readRequest(0xc42062e960, 0x1d72340, 0xc422c444c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: net/rpc.(*Server).ServeRequest(0xc42062e960, 0x1d72340, 0xc422c444c0, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4205e1400, 0x1d7d0e0, 0xc421e34240, 0xc42062e960) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 249 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc422c94400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 250 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b62c0, 0xc4204a3b00, 0x1d6eb80, 0xc422c94400, 0xc422ddd440, 0xc422ddd5c0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b62c0, 0xc422c94440) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 286 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422c44940, 0xc420061ce8, 0x20, 0x20, 0x18, 0xc422356be0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422944990, 0x1d71840, 0xc422c44940, 0xc422356be0, 0x184987) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4202d0b40, 0xc422bafed0, 0x184987, 0x1d71840, 0xc422c44940, 0xc420ddf260, 0xc4228e3770, 0xc420f8c960, 0x24, 0x2bba400) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420ddf200, 0x1d71840, 0xc422c44940, 0x184987, 0xc422d14f40, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420ddf200, 0x1d71840, 0xc422c44940) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 287 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422c449c0, 0xc422bab770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422944ab0, 0x1d71840, 0xc422c449c0, 0x0, 0x1849b5) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4202d0b40, 0xc422bab960, 0x1849b5, 0x1d71840, 0xc422c449c0, 0x552bde, 0xc422bab950, 0xc422bab970, 0xf33ba5, 0xc420230058) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc420230000, 0x1d71840, 0xc422c449c0, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc420230000) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 288 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422c44980, 0xc422bada78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4229edb90, 0x1d71840, 0xc422c44980, 0xc4229edce0, 0x1833a4) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4202d0b40, 0xc422badc68, 0x1833a4, 0x1d71840, 0xc422c44980, 0x552bde, 0xc42043c960, 0x2, 0xc421158100, 0x40) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422c44a00, 0x1833a4, 0x3b, 0xc422ccede8, 0x1, 0x1) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422c44a00) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 289 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422c44a40) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 290 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420fc6870, 0x1d71840, 0xc422c44980) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 251 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4229ed950, 0xc422ddd680) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 252 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4229ed950, 0xc422ddd6e0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: goroutine 253 [select]: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422d15040, 0xc422cced28, 0x20, 0x20, 0xc420056800, 0x1bd9a98) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4229ede00, 0x1d71840, 0xc422d15040, 0xc4228f46e0, 0x1849b5) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4202d0b40, 0xc422ab3f20, 0x1849b5, 0x1d71840, 0xc422d15040, 0x0, 0x0, 0x0, 0x0, 0xc42135cfae) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc420758000, 0x1849b5, 0xc42135cfb8, 0xc42135cee8, 0xc421a7f2c0, 0xf959ca, 0x1, 0x0) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc420758000, 0x1849b5, 0xc421a7f680) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25773]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:27:46 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:27:46 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:27:46 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: ==> Starting Nomad agent... Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: ==> Nomad agent configuration: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: Client: false Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: Log Level: INFO Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: Region: amrv01 (DC: test) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: Server: true Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: Version: 0.8.4 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:46 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:46 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:46 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:46 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:46.967259 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:46.968271 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:27:46 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:46 [INFO] serf: Attempting re-join to previously known node: consulnomad-01.aav01: 172.27.0.55:4648 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.322677 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.322700 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.322712 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.322724 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.322735 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [WARN] memberlist: Refuting an alive message Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323169 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323187 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323198 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323209 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47 [INFO] serf: Re-joined to previously known node: consulnomad-01.aav01: 172.27.0.55:4648 Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323561 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323573 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323585 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323597 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323607 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323619 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323630 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323641 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323652 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:27:47 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:47.323663 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:48 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:27:49 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:49.515335 [ERR] http: Request /v1/agent/health?type=server, error: {"server":{"ok":false,"message":"rpc error: failed to get conn: dial tcp 172.24.10.89:4647: connect: connection refused"}} Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 39 Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [INFO] raft: Election won. Tally: 2 Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50.623031 [INFO] nomad: cluster leadership acquired Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:50 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:50 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:27:51 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:51 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 501.470127ms Aug 23 15:27:51 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:51 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:51 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:51 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:51 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:51 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:51 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:51 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:51 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:51 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:51 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:51 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 983.714631ms Aug 23 15:27:51 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:51 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:52 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:52 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:52 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:52 [WARN] raft: Failed to contact afff500f-c899-4d26-d003-1a0d1d8bf516 in 1.421532612s Aug 23 15:27:52 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:52 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:52 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:52 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:53 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:53 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:53 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:53 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:54 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:54 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:55 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:55 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:27:57 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:27:57 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:28:01 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:01 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:28:02 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:02 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:28:11 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:11 [ERR] raft: Failed to AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:28:13 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:13 [ERR] raft: Failed to heartbeat to 172.24.0.121:4647: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:28:16 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:16 [WARN] raft: Rejecting vote request from 172.24.0.121:4647 since we have a leader: 172.24.20.236:4647 Aug 23 15:28:16 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:16 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:28:16 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:16.532950 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:28:17 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:17 [WARN] raft: Rejecting vote request from 172.24.0.121:4647 since we have a leader: 172.24.20.236:4647 Aug 23 15:28:18 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:18 [WARN] raft: Rejecting vote request from 172.24.0.121:4647 since we have a leader: 172.24.20.236:4647 Aug 23 15:28:20 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:20 [WARN] raft: Rejecting vote request from 172.24.0.121:4647 since we have a leader: 172.24.20.236:4647 Aug 23 15:28:21 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:21 [ERR] raft: peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} has newer term, stopping replication Aug 23 15:28:21 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:21 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:28:21 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:21 [INFO] raft: aborting pipeline replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:28:21 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:21.591044 [ERR] worker: failed to update evaluation : leadership lost while committing log Aug 23 15:28:21 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:21.591199 [INFO] nomad: cluster leadership lost Aug 23 15:28:22 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:22 [WARN] raft: Rejecting vote request from 172.24.0.121:4647 since our last term is greater (39, 11) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 44 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23 [INFO] raft: Election won. Tally: 2 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23.622155 [INFO] nomad: cluster leadership acquired Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23 [WARN] raft: AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} rejected, sending older logs (next: 1597365) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23.650731 [ERR] worker: failed to nack evaluation '2e2e8e1f-ac70-e04d-2342-fd9dde46a620': Evaluation ID not found Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: 2018/08/23 15:28:23.650778 [ERR] worker: failed to dequeue evaluation: eval broker disabled Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: fatal error: concurrent map writes Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3423 [running]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc422ae8d08 sp=0xc422ae8ce8 pc=0x42d161 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: runtime.mapdelete(0x1844be0, 0xc422ed65a0, 0xc422ae8e20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc422ae8d68 sp=0xc422ae8d08 pc=0x40a761 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc422e55cc0, 0x1d6dd40, 0xc422b79b48, 0x0, 0xed3109f67) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc422ae8e50 sp=0xc422ae8d68 pc=0xf2164c Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc420339040, 0x1d71840, 0xc422500fc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc422ae8fc8 sp=0xc422ae8e50 pc=0xf56bb7 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: runtime.goexit() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc422ae8fd0 sp=0xc422ae8fc8 pc=0x45c711 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42014d110, 0xc420422640) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42014d110, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4202b0640, 0xc4202b0640, 0xc4202ee820, 0x37) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204e7f68) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc42009e058) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: main.main() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 5 [syscall]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: os/signal.signal_recv(0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: os/signal.loop() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by os/signal.init.0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 171 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420240160, 0xc42122fd10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420240160, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc42000e510, 0xc421068050) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc42000e510, 0xc421068050) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 173 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c08036e0, 0x72, 0xc421260ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc421285018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc421285018, 0xc42161e000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc421285000, 0xc42161e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc421285000, 0xc42161e000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4202184c0, 0xc42161e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc420558e40, 0xc422c14730, 0xc, 0xc, 0x3, 0x40df9e, 0xc421162180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc420558e40, 0xc422c14730, 0xc, 0xc, 0xc, 0xc, 0xc4224406c0, 0xc4215e4044) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc420558e40, 0xc422c14730, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4215e4000, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4215e4000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 58 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4207403c0, 0x12a05f200, 0xc4214f2780, 0xc420eda180, 0xc42068e230) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 53 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4200b7810, 0x1c636a0, 0xc4200cc2c0, 0x1d71840, 0xc420206fc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4200cc2c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 121 [runnable]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt._assert(0x19e6b01, 0x1c03f69, 0x1b, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt/db.go:1027 +0xfd Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt.(*node).read(0xc422f9d880, 0x7fe8a3051000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt/node.go:178 +0xcd Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt.(*Bucket).node(0xc423808018, 0x3051, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt/bucket.go:666 +0x12d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt.(*Cursor).node(0xc4214df558, 0x4) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt/cursor.go:369 +0x1b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt.(*Bucket).spill(0xc423808018, 0x89604ff26, 0x2b96de0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt/bucket.go:561 +0x2db Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt.(*Tx).Commit(0xc423808000, 0xc422fa22a8, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/boltdb/bolt/tx.go:163 +0x11f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft-boltdb.(*BoltStore).StoreLogs(0xc420527b80, 0xc4220ddbb0, 0x1, 0x1, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft-boltdb/bolt_store.go:157 +0x26b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*LogCache).StoreLogs(0xc420207080, 0xc4220ddbb0, 0x1, 0x1, 0x1, 0x1) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/log_cache.go:61 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).dispatchLogs(0xc42056c580, 0xc4214dfb10, 0x1, 0x1) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:856 +0x29d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc42056c580) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:599 +0x73a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc42056c580) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc42056c580) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42056c580, 0xc420665af0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 45 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc42042d400, 0x1d71840, 0xc420214f80) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 174 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4215e4000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 38 [select, locked to thread]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: runtime.selectgo(0xc4201b7f50, 0xc420150420) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: runtime.ensureSigM.func1() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: runtime.goexit() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 39 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42042d340) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 40 [sleep]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: time.Sleep(0x3b9aca00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420076c60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 41 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420154540) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 42 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc42044cc00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 197 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420c81b80) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 44 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc420380270) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0xc4201b3208, 0x502a15) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4201bb440, 0xc420474c60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 24 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803f00, 0x72, 0xc4200619a8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc420142918, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc420142918, 0xc420564000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc420142900, 0xc420564000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc420142900, 0xc420564000, 0x1000, 0x1000, 0x458600, 0xc4209b6a80, 0x4) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4202101c8, 0xc420564000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*persistConn).Read(0xc420406d80, 0xc420564000, 0x1000, 0x1000, 0xc420061b98, 0x405675, 0xc4202f2240) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc420086c60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Peek(0xc420086c60, 0x1, 0x0, 0x0, 0x1, 0xc4218396e0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*persistConn).readLoop(0xc420406d80) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by net/http.(*Transport).dialConn Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 196 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803470, 0x72, 0xc420514ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc422ae4898, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc422ae4898, 0xc4211a8000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc422ae4880, 0xc4211a8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc422ae4880, 0xc4211a8000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420218b90, 0xc4211a8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc420523e60, 0xc421f66840, 0xc, 0xc, 0x3, 0x40df9e, 0xc4214f2180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc420523e60, 0xc421f66840, 0xc, 0xc, 0xc, 0xc, 0xc422a48c00, 0xc420c81bc4) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc420523e60, 0xc421f66840, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420c81b80, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420c81b80) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 59 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4207403c0, 0xc420eda180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:143 +0x1f3 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 25 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*persistConn).writeLoop(0xc420406d80) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by net/http.(*Transport).dialConn Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 170 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420244240, 0xc42113c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4210236e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4210236e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4210fc1e0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420642ea0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421023740, 0x16e4b60, 0xc420642f00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421023740, 0x16e4b60, 0xc420642f00, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421117380, 0x16e4b60, 0xc420642f00, 0xc420642f00, 0xc421186528) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421117380, 0xc420642f00, 0x2d, 0xc4219daac0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc421186500, 0x1d72340, 0xc421117380, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc421186500, 0x1d72340, 0xc421117380, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421186500, 0x1d72340, 0xc421117380, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc420244240, 0xc421186500) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 60 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4207403c0, 0x1dcd6500, 0xc4214f27e0, 0xc420eda180, 0xc42068e240) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 175 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4215e4000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 219 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4202443c0, 0xc420ae6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc420eb4840) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc420eb4840, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4212f2f00, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420f6e240) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420eb48a0, 0x16e4b60, 0xc420f6e2a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420eb48a0, 0x16e4b60, 0xc420f6e2a0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421f07a40, 0x16e4b60, 0xc420f6e2a0, 0xc420f6e2a0, 0xc4214eda68) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421f07a40, 0xc420f6e2a0, 0x2d, 0xc421fc1ca0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc4214eda40, 0x1d72340, 0xc421f07a40, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc4214eda40, 0x1d72340, 0xc421f07a40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4214eda40, 0x1d72340, 0xc421f07a40, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc4202443c0, 0xc4214eda40) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 216 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803af0, 0x72, 0xc42127ace8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc422ae4818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc422ae4818, 0xc420fdf000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc422ae4800, 0xc420fdf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc422ae4800, 0xc420fdf000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420218b88, 0xc420fdf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc4212831a0, 0xc421d8b850, 0xc, 0xc, 0x3, 0x40df9e, 0xc420086180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc4212831a0, 0xc421d8b850, 0xc, 0xc, 0xc, 0xc, 0xc42238cd80, 0xc4215e43b4) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc4212831a0, 0xc421d8b850, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4215e4370, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4215e4370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 195 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4215e4160, 0xc421e16d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4215e4160, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420210090, 0xc4214ed9a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420210090, 0xc4214ed9a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 226 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4215e4160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 122 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc42056c580) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42056c580, 0xc420665b00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 123 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc42056c580) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42056c580, 0xc420665b60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 124 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc4214d1f20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 125 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4206e01a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 126 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803e30, 0x72, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc420771098, 0x72, 0xc420ee8300, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc420771098, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Accept(0xc420771080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).accept(0xc420771080, 0xc420219740, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*TCPListener).accept(0xc42000e658, 0x458e40, 0xc4201b1f90, 0xc4201b1f98) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*TCPListener).AcceptTCP(0xc42000e658, 0x1c63658, 0xc4207403c0, 0xc420219740) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4207403c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 127 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803c90, 0x72, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc420771b18, 0x72, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc420771b18, 0xc42375a000, 0x10000, 0x10000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).ReadFrom(0xc420771b00, 0xc42375a000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).readFrom(0xc420771b00, 0xc42375a000, 0x10000, 0x10000, 0x96, 0x96, 0x1d62b40, 0x412539, 0xc4234ffe40) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*UDPConn).readFrom(0xc42000e670, 0xc42375a000, 0x10000, 0x10000, 0x101, 0xc4234ffe18, 0xc4234ffe90, 0x443cf7) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*UDPConn).ReadFrom(0xc42000e670, 0xc42375a000, 0x10000, 0x10000, 0x10000, 0x10000, 0xe17a, 0x83afe513c, 0x2b96de0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4207403c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 128 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4207403c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 61 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc421384480) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 62 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc421384480) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 63 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421384480, 0x1bdd09a, 0x6, 0xc4214ee090) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 64 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421384480, 0x1bdba6c, 0x5, 0xc4214ee0c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 65 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421384480, 0x1bdbc6a, 0x5, 0xc4214ee0f0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 217 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4215e4370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 131 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420286f00, 0x1be6698, 0xc, 0x1d76b80, 0xc422f79740, 0x19746e0, 0xc422f79740, 0x181f280, 0xc422c41700, 0xc42044d6e0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc42000e9e0, 0xc422f79740, 0xc422c41700, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc42044d6e0, 0xc42000ee80, 0x13, 0x1bda9a0, 0x4, 0xc42333ed18, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc42044d6e0, 0xc42000ee80, 0x13, 0xc42333ed18, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc42042d540, 0xc42020b540, 0xc422f36818, 0x0, 0xc420396200, 0xc4214e8240, 0x19746e0, 0xc422f79740, 0x16, 0x181f280, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc42020b540, 0x1d71b80, 0xc422c41600, 0x5b7ea865, 0xc4233fba0c) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420286f00, 0x1be6698, 0xc, 0x19746e0, 0xc4214e6d80, 0x181f280, 0xc42124c180, 0x1d59ee0, 0xc4206f0d80) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214f2840, 0x1dcd6500, 0x0, 0x0, 0x0, 0xc42009e420, 0x71a2da) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:158 +0x1bb Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214f2840) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 132 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420286f00, 0x1be6698, 0xc, 0x1d76b80, 0xc422f79800, 0x19746e0, 0xc422f79800, 0x181f280, 0xc422c41780, 0xc42044d6e0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc42000e9e0, 0xc422f79800, 0xc422c41780, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc42044d6e0, 0xc42000ee80, 0x13, 0x1bda9a0, 0x4, 0xc42333fd18, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc42044d6e0, 0xc42000ee80, 0x13, 0xc42333fd18, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc42042d540, 0xc42020b540, 0xc422f36820, 0x0, 0xc420396200, 0xc422e55ea0, 0x19746e0, 0xc422f79800, 0x16, 0x181f280, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc42020b540, 0x1d71b80, 0xc422c41740, 0x5b7ea865, 0xc4233fe2f6) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420286f00, 0x1be6698, 0xc, 0x19746e0, 0xc4214e6f60, 0x181f280, 0xc42124c240, 0x1d59ee0, 0xc422b96f50) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214f28a0, 0x1dcd6500, 0x63c900, 0x1, 0x0, 0x435248, 0x1c66060) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:158 +0x1bb Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214f28a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 133 [semacquire]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: sync.runtime_notifyListWait(0xc421248110, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/sema.go:510 +0x10b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: sync.(*Cond).Wait(0xc421248100) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/sync/cond.go:56 +0x80 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Worker).checkPaused(0xc4214f2900) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:100 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4214f2900, 0x1dcd6500, 0xc420df0120, 0x9ebabe, 0x1, 0x0, 0x435248) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:154 +0x141 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214f2900) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 134 [chan receive]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*deferError).Error(0xc4237d7380, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/future.go:100 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).raftApply(0xc420286f00, 0xc42036ec06, 0x1974920, 0xc4238060c0, 0x24, 0x0, 0x0, 0x2b96de0, 0xc422fa2260) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:575 +0x83 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Eval).Update(0xc42000e9e0, 0xc4238060c0, 0xc422fa2260, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:204 +0x21a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc42044d680, 0xc42000eef8, 0x13, 0x1bda9a0, 0x4, 0xc421183a68, 0x3, 0x3, 0x18deac0, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc42044d680, 0xc42000eef8, 0x13, 0xc421183a68, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc42042d540, 0xc42020b540, 0xc422fa2258, 0x0, 0xc420396500, 0xc4214e8200, 0x1974920, 0xc4238060c0, 0x16, 0x169db60, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc42020b540, 0x1d71b80, 0xc421b14500, 0x18deac0, 0x1) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420286f00, 0x1be4dcd, 0xb, 0x1974920, 0xc423806060, 0x169db60, 0xc422fa2250, 0x2b96de0, 0xc42020b2c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Worker).UpdateEval(0xc4214f2960, 0xc423798580, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:388 +0x2ad Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/scheduler.setStatus(0xc42020b2c0, 0x1d71c00, 0xc4214f2960, 0xc4202bc6e0, 0x0, 0x0, 0x0, 0x1be0dd6, 0x8, 0x0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/scheduler/util.go:455 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/scheduler.(*GenericScheduler).Process(0xc4223d2dc0, 0xc4202bc6e0, 0xc42020b2c0, 0x1d7cde0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/scheduler/generic_sched.go:174 +0x7b3 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Worker).invokeScheduler(0xc4214f2960, 0xc4202bc6e0, 0xc422e50090, 0x24, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:292 +0x433 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4214f2960) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:127 +0x12a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 135 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*persistConn).roundTrip(0xc420406d80, 0xc4221ffd10, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/transport.go:2033 +0x5a7 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*Transport).RoundTrip(0xc4203282d0, 0xc4208bd500, 0xc4203282d0, 0xbed7c7fb2612dbbc, 0x9be1788f2) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/transport.go:422 +0x8f2 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.send(0xc4208bd400, 0x1d5c2c0, 0xc4203282d0, 0xbed7c7fb2612dbbc, 0x9be1788f2, 0x2b96de0, 0xc420556dd0, 0xbed7c7fb2612dbbc, 0xc4212eb900, 0x1) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/client.go:252 +0x185 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*Client).send(0xc4201bab70, 0xc4208bd400, 0xbed7c7fb2612dbbc, 0x9be1788f2, 0x2b96de0, 0xc420556dd0, 0x0, 0x1, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/client.go:176 +0xfa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*Client).Do(0xc4201bab70, 0xc4208bd400, 0x2b96de0, 0x0, 0xb40000c4212ebaf8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/client.go:615 +0x28d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/api.(*Client).doRequest(0xc420154480, 0xc4237bd800, 0x1bda383, 0x3, 0xc42316c7c8, 0xc4237bd800) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/api/api.go:673 +0xb8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/api.(*Catalog).Service(0xc42000e988, 0x1bdc561, 0x5, 0x1bdb4a4, 0x4, 0xc4237aff10, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/api/catalog.go:164 +0x12c Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc42125bf40, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:804 +0x5c7 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc4214f4270, 0xc420286f00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 136 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc420286f00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 137 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc420286f00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 138 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803d60, 0x72, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc420530918, 0x72, 0xc420554000, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc420530918, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Accept(0xc420530900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).accept(0xc420530900, 0x1, 0x1, 0xc421261f20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*TCPListener).accept(0xc420556238, 0x406b4a, 0xc420f52b40, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*TCPListener).Accept(0xc420556238, 0xc420f52b40, 0x0, 0x3f800000, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 139 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc420339040, 0x3b9aca00, 0xc4201508a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 140 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc42020b4f0, 0x3b9aca00, 0xc4201508a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 141 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420152630, 0x3b9aca00, 0xc4201508a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 142 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc420380270, 0x3b9aca00, 0xc4201508a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 143 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc420286f00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 144 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803bc0, 0x72, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc420b28518, 0x72, 0xc420750000, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc420b28518, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Accept(0xc420b28500, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).accept(0xc420b28500, 0x402037, 0xc4211e0440, 0x1805ec0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*TCPListener).accept(0xc420556060, 0x458e40, 0xc42127be90, 0xc42127be98) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*TCPListener).Accept(0xc420556060, 0x1c65af8, 0xc4211e03c0, 0x1d71900, 0xc421265c20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*Server).Serve(0xc42059dee0, 0x1d6fc00, 0xc420556060, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.Serve(0x1d6fc00, 0xc420556060, 0x1d5df00, 0xc421265b60, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc421248240, 0x1d6fc00, 0xc420556060, 0xc42068e7d0, 0xc4214f4330) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 145 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420b28580, 0xc42068e7f0, 0xc420eda2a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 218 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4215e4370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 147 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803880, 0x72, 0xc420faace8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc420a83698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc420a83698, 0xc420cba000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc420a83680, 0xc420cba000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc420a83680, 0xc420cba000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4213a6000, 0xc420cba000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc42105b200, 0xc420f73720, 0xc, 0xc, 0x3, 0x40df9e, 0xc4213c5620) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc42105b200, 0xc420f73720, 0xc, 0xc, 0xc, 0xc, 0xc420d6f080, 0xc4202400f4) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc42105b200, 0xc420f73720, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202400b0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202400b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 199 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421270e40, 0xc4211a9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4210a8b40) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4210a8b40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4210265d0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421901e60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210a8ba0, 0x16e4b60, 0xc421901ee0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210a8ba0, 0x16e4b60, 0xc421901ee0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421f8b0c0, 0x16e4b60, 0xc421901ee0, 0xc421901ee0, 0xc420dc5dd8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421f8b0c0, 0xc421901ee0, 0x2d, 0xc421c2eca0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc420dc5db0, 0x1d72340, 0xc421f8b0c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc420dc5db0, 0x1d72340, 0xc421f8b0c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc420dc5db0, 0x1d72340, 0xc421f8b0c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421270e40, 0xc420dc5db0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 172 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4215e4000, 0xc420fabd10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4215e4000, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4202184c0, 0xc42200e190) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4202184c0, 0xc42200e190) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 198 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420c81b80) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 184 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202400b0, 0xc420fa9d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202400b0, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4213a6000, 0xc4214724b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4213a6000, 0xc4214724b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 148 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202400b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 149 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202400b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 150 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421270480, 0xc420691000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4213c5920) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4213c5920, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4213ca750, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421110e40) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4213c5980, 0x16e4b60, 0xc421110ea0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4213c5980, 0x16e4b60, 0xc421110ea0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421117100, 0x16e4b60, 0xc421110ea0, 0xc421110ea0, 0xc421063158) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421117100, 0xc421110ea0, 0x2d, 0xc4219113c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc421063130, 0x1d72340, 0xc421117100, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc421063130, 0x1d72340, 0xc421117100, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421063130, 0x1d72340, 0xc421117100, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421270480, 0xc421063130) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 185 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420298dc0, 0xc420066d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420298dc0, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc42000e028, 0xc42043c370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc42000e028, 0xc42043c370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 186 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c08037b0, 0x72, 0xc420067ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc42108c018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc42108c018, 0xc420b8b000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc42108c000, 0xc420b8b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc42108c000, 0xc420b8b000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc42000e028, 0xc420b8b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc42149bc80, 0xc420a31890, 0xc, 0xc, 0x3, 0x40df9e, 0xc421b806c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc42149bc80, 0xc420a31890, 0xc, 0xc, 0xc, 0xc, 0xc420bc5c80, 0xc420298e04) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc42149bc80, 0xc420a31890, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420298dc0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420298dc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 187 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420298dc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 188 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420298dc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 189 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421023c20, 0xc421181700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420ffd3b0, 0x1d718c0, 0xc421023c20, 0x0, 0xc421023c00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420286f00, 0xc421181950, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421099a40, 0xc4211a2000, 0xc420ffd260, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc420558a20, 0xc42000fd98, 0x13, 0x1bda9a0, 0x4, 0xc421181d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc420558a20, 0xc42000fd98, 0x13, 0xc421181d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc42109e2c0, 0xc42043c410, 0xc420a319c8, 0x0, 0xc4210a0900, 0xc420d28d20, 0x19758e0, 0xc4211a2000, 0x16, 0x169e020, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc42043c410, 0x1d72340, 0xc42109e380, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421270300, 0xc42043c410) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 190 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42119e000, 0xc420fe9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4210a8420) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4210a8420, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4210a4300, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420d28e80) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210a8480, 0x16e4b60, 0xc420d28ee0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210a8480, 0x16e4b60, 0xc420d28ee0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42109e500, 0x16e4b60, 0xc420d28ee0, 0xc420d28ee0, 0xc42043c438) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42109e500, 0xc420d28ee0, 0x2d, 0xc422e66340) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc42043c410, 0x1d72340, 0xc42109e500, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc42043c410, 0x1d72340, 0xc42109e500, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc42043c410, 0x1d72340, 0xc42109e500, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119e000, 0xc42043c410) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 151 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420298f20, 0xc42127cd10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420298f20, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420210170, 0xc420217220) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420210170, 0xc420217220) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 191 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803610, 0x72, 0xc42050fce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc4210a0e98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc4210a0e98, 0xc420a4a000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc4210a0e80, 0xc420a4a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc4210a0e80, 0xc420a4a000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420210170, 0xc420a4a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc420eb4b40, 0xc42074bc30, 0xc, 0xc, 0x3, 0x40df9e, 0xc420e86300) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc420eb4b40, 0xc42074bc30, 0xc, 0xc, 0xc, 0xc, 0xc4210748a0, 0xc420298f64) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc420eb4b40, 0xc42074bc30, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420298f20, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420298f20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 192 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420298f20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 193 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420298f20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 194 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4210234a0, 0xc4211fb700, 0x20, 0x20, 0x169d4e0, 0xc42001a280) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4210fc000, 0x1d718c0, 0xc4210234a0, 0x0, 0xc421023400) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420286f00, 0xc4211fb950, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42018d620, 0xc4213ccd80, 0xc4213bf680, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc420558a20, 0xc4205565a8, 0x13, 0x1bda9a0, 0x4, 0xc42122ed48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc420558a20, 0xc4205565a8, 0x13, 0xc42122ed48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc42051ee40, 0xc421186500, 0xc420a62110, 0x0, 0xc421284700, 0xc4205269e0, 0x19758e0, 0xc4213ccd80, 0x16, 0x169e020, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421186500, 0x1d72340, 0xc42051ee80, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc420244180, 0xc421186500) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 210 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803540, 0x72, 0xc421e18ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc421284e18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc421284e18, 0xc4211af000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc421284e00, 0xc4211af000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc421284e00, 0xc4211af000, 0x1000, 0x1000, 0xac, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc42000e510, 0xc4211af000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc42190cc00, 0xc421a7e060, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc42190cc00, 0xc421a7e060, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc42190cc00, 0xc421a7e060, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420240160, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420240160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 211 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420240160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 212 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420240160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 213 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420eb4a80, 0xc421201700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420ca6f90, 0x1d718c0, 0xc420eb4a80, 0x0, 0xc420eb4a00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420286f00, 0xc421201950, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421a5cb40, 0xc421941c00, 0xc420ca6e40, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc420558a20, 0xc4213a6340, 0x13, 0x1bda9a0, 0x4, 0xc421201d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc420558a20, 0xc4213a6340, 0x13, 0xc421201d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc421d1e000, 0xc421ec14f0, 0xc421a7e378, 0x0, 0xc4211a2e80, 0xc421ee52c0, 0x19758e0, 0xc421941c00, 0x16, 0x169e020, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421ec14f0, 0x1d72340, 0xc421d1e040, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119e180, 0xc421ec14f0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 177 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803950, 0x72, 0xc42050ece8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc4213cd298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc4213cd298, 0xc421323000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc4213cd280, 0xc421323000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc4213cd280, 0xc421323000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420210090, 0xc421323000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc420087e00, 0xc421d8b7d0, 0xc, 0xc, 0x3, 0x40df9e, 0xc420523c20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc420087e00, 0xc421d8b7d0, 0xc, 0xc, 0xc, 0xc, 0xc42277b920, 0xc4215e41a4) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc420087e00, 0xc421d8b7d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4215e4160, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4215e4160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 227 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4215e4160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 307 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421824160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3419 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc42056c580, 0xc4221e6780, 0x1d6eb80, 0xc422500480, 0xc4226af860, 0xc4226af8c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42056c580, 0xc4225004c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 233 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803a20, 0x72, 0xc422a56ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc420770a18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc420770a18, 0xc420fba000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc420770a00, 0xc420fba000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc420770a00, 0xc420fba000, 0x1000, 0x1000, 0x68, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420218b80, 0xc420fba000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc4214e7e60, 0xc422c14a40, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc4214e7e60, 0xc422c14a40, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc4214e7e60, 0xc422c14a40, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4215e4210, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4215e4210) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 153 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4215e4370, 0xc42122bd10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4215e4370, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420218b88, 0xc42200e780) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420218b88, 0xc42200e780) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 154 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420c81b80, 0xc42122ed10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420c81b80, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420218b90, 0xc42200e8c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420218b90, 0xc42200e8c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 234 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4215e4210) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 235 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4215e4210) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 306 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803060, 0x72, 0xc422a63ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc422ae5098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc422ae5098, 0xc421cb0000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc422ae5080, 0xc421cb0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc422ae5080, 0xc421cb0000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420218d40, 0xc421cb0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc421b81200, 0xc421cfdfd4, 0xc, 0xc, 0x3, 0x40df9e, 0xc421a5d920) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc421b81200, 0xc421cfdfd4, 0xc, 0xc, 0xc, 0xc, 0xc4228efda0, 0xc4218241a4) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc421b81200, 0xc421cfdfd4, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421824160, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421824160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3420 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420718aa0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3449 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4227782a0, 0xc4222bc000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 205 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4215e4580, 0xc422a55d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4215e4580, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4205575e8, 0xc4218004b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4205575e8, 0xc4218004b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 223 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c08033a0, 0x72, 0xc421e19ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc421245298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc421245298, 0xc420010000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc421245280, 0xc420010000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc421245280, 0xc420010000, 0x1000, 0x1000, 0xc421e19dd8, 0x1, 0xc421e19e08) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4205575e8, 0xc420010000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc420eb5260, 0xc421051bd4, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc420eb5260, 0xc421051bd4, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc420eb5260, 0xc421051bd4, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4215e4580, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4215e4580) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 224 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4215e4580) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 225 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4215e4580) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3417 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42119f500, 0xc42379a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc422f832c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc422f832c0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4223179e0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422373fa0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422f83320, 0x16e4b60, 0xc422ef0000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422f83320, 0x16e4b60, 0xc422ef0000, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422500200, 0x16e4b60, 0xc422ef0000, 0xc422ef0000, 0xc4214edc98) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422500200, 0xc422ef0000, 0x2d, 0xc422f94360) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc4214edc70, 0x1d72340, 0xc422500200, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc4214edc70, 0x1d72340, 0xc422500200, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4214edc70, 0x1d72340, 0xc422500200, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119f500, 0xc4214edc70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 258 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42119fd40, 0xc421478000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4216565a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4216565a0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420fb8840, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4214e8e00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421656660, 0x16e4b60, 0xc4214e8e80) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421656660, 0x16e4b60, 0xc4214e8e80, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422018940, 0x16e4b60, 0xc4214e8e80, 0xc4214e8e80, 0xc42200e258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422018940, 0xc4214e8e80, 0x2d, 0xc4222ed760) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc42200e230, 0x1d72340, 0xc422018940, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc42200e230, 0x1d72340, 0xc422018940, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc42200e230, 0x1d72340, 0xc422018940, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119fd40, 0xc42200e230) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3450 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4227782a0, 0xc4222bc1e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 264 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420244fc0, 0xc422324000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4225d7aa0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4225d7aa0, 0xc420040000, 0xc422398ad8, 0xc422398ae0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421c77cb0, 0xc4212e9990) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421a79580) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4225d7b60, 0x16e4ba0, 0xc421cf8090) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4225d7b60, 0x16e4ba0, 0xc421cf8090, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4221d2c80, 0x16e4ba0, 0xc421cf8090, 0x1, 0xc421cf8090) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc4221d2c80, 0xc421cf8090, 0x1972ee0, 0xc42107a240) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc4221d2c80, 0x1beced6, 0x10, 0x1972ee0, 0xc42107a240, 0x169d620, 0xc421c77bf0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc42044cc00, 0xc4201f4a29, 0x6, 0x1d62b00, 0xc420b618f0, 0x1, 0x1beced6, 0x10, 0x1972ee0, 0xc42107a240, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).forwardRegion(0xc420286f00, 0xc4201f4a29, 0x6, 0x1beced6, 0x10, 0x1972ee0, 0xc42107a240, 0x169d620, 0xc421c77bf0, 0x0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:465 +0x232 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).replicateACLPolicies(0xc420286f00, 0xc421704000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:926 +0x2cb Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:267 +0x658 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 265 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42119ef00, 0xc4220ff000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4219bdce0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4219bdce0, 0xc420046f00, 0xc420b36318, 0xc420b36320) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420f5c510, 0xc42264b988) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42076e4e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4219bdd40, 0x16e4ba0, 0xc420f5cd20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4219bdd40, 0x16e4ba0, 0xc420f5cd20, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420cd7040, 0x16e4ba0, 0xc420f5cd20, 0x1, 0xc420f5cd20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc420cd7040, 0xc420f5cd20, 0x1973360, 0xc42048b810) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc420cd7040, 0x1be96aa, 0xe, 0x1973360, 0xc42048b810, 0x169d6e0, 0xc42112f260, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc42044cc00, 0xc4201f4a29, 0x6, 0x1d62b00, 0xc420b60e70, 0x1, 0x1be96aa, 0xe, 0x1973360, 0xc42048b810, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).forwardRegion(0xc420286f00, 0xc4201f4a29, 0x6, 0x1be96aa, 0xe, 0x1973360, 0xc42048b810, 0x169d6e0, 0xc42112f260, 0x0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:465 +0x232 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).replicateACLTokens(0xc420286f00, 0xc421704000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:1068 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:268 +0x68a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 292 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c0803130, 0x72, 0xc4224e7ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc421245998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc421245998, 0xc421d00000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc421245980, 0xc421d00000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc421245980, 0xc421d00000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420557ef8, 0xc421d00000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc4222845a0, 0xc4225f2e40, 0xc, 0xc, 0xc, 0x436544, 0xc4224e7e98) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc4222845a0, 0xc4225f2e40, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc4222845a0, 0xc4225f2e40, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420994a50, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420994a50) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3440 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420286f00, 0x1be4dcd, 0xb, 0x1d76c40, 0xc422faa7e0, 0x1974920, 0xc422faa7e0, 0x169db60, 0xc422f8ea58, 0xc422faa808, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Eval).Update(0xc42000e9e0, 0xc422faa7e0, 0xc422f8ea58, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:187 +0xa6 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc42044d680, 0xc420210668, 0x13, 0x1bda9a0, 0x4, 0xc4227a7d48, 0x3, 0x3, 0x18deac0, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc42044d680, 0xc420210668, 0x13, 0xc4227a7d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc421fd2180, 0xc4214edc70, 0xc422f8e6a0, 0x0, 0xc421021680, 0xc422e12440, 0x1974920, 0xc422faa7e0, 0x16, 0x169db60, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4214edc70, 0x1d72340, 0xc422cf8c80, 0x0, 0xc42119ecc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119ecc0, 0xc4214edc70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 322 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420286f00, 0x1be6698, 0xc, 0x1d76b80, 0xc422f82000, 0x19746e0, 0xc422f82000, 0x181f280, 0xc422ccd440, 0xc42044d6e0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc42000e9e0, 0xc422f82000, 0xc422ccd440, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc42044d6e0, 0xc420210638, 0x13, 0x1bda9a0, 0x4, 0xc421291d48, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc42044d6e0, 0xc420210638, 0x13, 0xc421291d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc421fd2180, 0xc4214edc70, 0xc423465bf8, 0x0, 0xc421021380, 0xc421017ca0, 0x19746e0, 0xc422f82000, 0x16, 0x181f280, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4214edc70, 0x1d72340, 0xc422482080, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119fec0, 0xc4214edc70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 293 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420994a50) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 294 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420994a50) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 308 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421824160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 295 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420994b00, 0xc422a67d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420994b00, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420557f00, 0xc42200fc20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420557f00, 0xc42200fc20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 296 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f858, 0x72, 0xc421e15ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc420f71418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc420f71418, 0xc421d01000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc420f71400, 0xc421d01000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc420f71400, 0xc421d01000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420557f00, 0xc421d01000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc4222848a0, 0xc4225b0200, 0xc, 0xc, 0x3, 0x40df9e, 0xc4205580c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc4222848a0, 0xc4225b0200, 0xc, 0xc, 0xc, 0xc, 0xc4228db500, 0xc420994b44) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc4222848a0, 0xc4225b0200, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420994b00, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420994b00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 297 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420994b00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 298 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420994b00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 299 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420155b00, 0xc421574000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4225d6fc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4225d6fc0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421c5c630, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421a78820) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4225d7020, 0x16e4b60, 0xc421a788a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4225d7020, 0x16e4b60, 0xc421a788a0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4223c5800, 0x16e4b60, 0xc421a788a0, 0xc421a788a0, 0xc42200fce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4223c5800, 0xc421a788a0, 0x2d, 0xc422abe140) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc42200fcc0, 0x1d72340, 0xc4223c5800, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc42200fcc0, 0x1d72340, 0xc4223c5800, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc42200fcc0, 0x1d72340, 0xc4223c5800, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc420155b00, 0xc42200fcc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2734 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420994210, 0xc422926d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420994210, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc42000e728, 0xc422450910) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc42000e728, 0xc422450910) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3083 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420c80c60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2888 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072ea88, 0x72, 0xc42292aac8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc42151c918, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc42151c918, 0xc421dd8000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc42151c900, 0xc421dd8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc42151c900, 0xc421dd8000, 0x1000, 0x1000, 0xc421cbb0e0, 0xc42292abe0, 0x7b9b0e) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4202183d0, 0xc421dd8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc42215b620) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc42215b620, 0x3, 0xc420042570, 0xc420042500) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleCommand(0xc4200cc2c0, 0xc42215b620, 0xc42215b680, 0xc42126e460, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:515 +0x43 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleConn(0xc4200cc2c0, 0x1d71840, 0xc420206fc0, 0x1d7d320, 0xc4202183d0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:499 +0x23d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:477 +0x1ce Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 358 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4218248f0, 0xc4224edd10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4218248f0, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420218ea0, 0xc422610ff0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420218ea0, 0xc422610ff0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 368 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420286f00, 0x1be6698, 0xc, 0x1d76b80, 0xc422f79920, 0x19746e0, 0xc422f79920, 0x181f280, 0xc422c418c0, 0xc42044d6e0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc42000e9e0, 0xc422f79920, 0xc422c418c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc42044d6e0, 0xc420210638, 0x13, 0x1bda9a0, 0x4, 0xc42264dd48, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc42044d6e0, 0xc420210638, 0x13, 0xc42264dd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc421fd2180, 0xc4214edc70, 0xc421bee2d8, 0x0, 0xc421021380, 0xc421c93840, 0x19746e0, 0xc422f79920, 0x16, 0x181f280, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4214edc70, 0x1d72340, 0xc421efa7c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421271d40, 0xc4214edc70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 385 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421824e70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 359 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f788, 0x72, 0xc422a64ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc422990198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc422990198, 0xc4229c3000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc422990180, 0xc4229c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc422990180, 0xc4229c3000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420218ea0, 0xc4229c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc42190cfc0, 0xc422aa2f00, 0xc, 0xc, 0x3, 0x40df9e, 0xc422b74ae0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc42190cfc0, 0xc422aa2f00, 0xc, 0xc, 0xc, 0xc, 0xc4218389c0, 0xc421824934) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc42190cfc0, 0xc422aa2f00, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4218248f0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4218248f0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 482 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421271ec0, 0xc4213d9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4223c6360) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4223c6360, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422375f80, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4202ee0a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4223c63c0, 0x16e4b60, 0xc4202ee140) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4223c63c0, 0x16e4b60, 0xc4202ee140, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4227ca6c0, 0x16e4b60, 0xc4202ee140, 0xc4202ee140, 0xc421f1c0c8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4227ca6c0, 0xc4202ee140, 0x2d, 0xc422d68ae0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc421f1c0a0, 0x1d72340, 0xc4227ca6c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc421f1c0a0, 0x1d72340, 0xc4227ca6c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421f1c0a0, 0x1d72340, 0xc4227ca6c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421271ec0, 0xc421f1c0a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 362 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42107bb00, 0xc4227a1700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421c8daa0, 0x1d718c0, 0xc42107bb00, 0x0, 0xc42107bb00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420286f00, 0xc4227a1950, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42215aea0, 0xc421245f00, 0xc421c8d7a0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc420558a20, 0xc4202191a8, 0x13, 0x1bda9a0, 0x4, 0xc422a50d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc420558a20, 0xc4202191a8, 0x13, 0xc422a50d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc422283000, 0xc422611090, 0xc422578d18, 0x0, 0xc421174a80, 0xc421d17ee0, 0x19758e0, 0xc421245f00, 0x16, 0x169e020, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc422611090, 0x1d72340, 0xc422283040, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421271c80, 0xc422611090) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 332 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4221e6000, 0xc4220f9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc42107b860) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc42107b860, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421c8cb70, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421c926e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42107b8c0, 0x16e4b60, 0xc421c92740) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42107b8c0, 0x16e4b60, 0xc421c92740, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422482600, 0x16e4b60, 0xc421c92740, 0xc421c92740, 0xc4226110b8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422482600, 0xc421c92740, 0x2d, 0xc4221a8b80) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc422611090, 0x1d72340, 0xc422482600, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc422611090, 0x1d72340, 0xc422482600, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc422611090, 0x1d72340, 0xc422482600, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc4221e6000, 0xc422611090) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2401 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42119f080, 0xc421b8d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc421ea4e40) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc421ea4e40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4216ddad0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420f68880) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421ea4ea0, 0x16e4b60, 0xc420f688e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421ea4ea0, 0x16e4b60, 0xc420f688e0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421d3b940, 0x16e4b60, 0xc420f688e0, 0xc420f688e0, 0xc4222ae848) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421d3b940, 0xc420f688e0, 0x2d, 0xc421ed8d20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc4222ae820, 0x1d72340, 0xc421d3b940, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc4222ae820, 0x1d72340, 0xc421d3b940, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4222ae820, 0x1d72340, 0xc421d3b940, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119f080, 0xc4222ae820) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1867 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420994370, 0xc422928d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420994370, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4220dd238, 0xc420dc44b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4220dd238, 0xc420dc44b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 361 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4218248f0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2708 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc423490840) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 360 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4218248f0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1184 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421db8e70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1016 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420c81ef0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 383 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f518, 0x72, 0xc4224ebce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc4215a1b18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc4215a1b18, 0xc4213d8000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc4215a1b00, 0xc4213d8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc4215a1b00, 0xc4213d8000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420211198, 0xc4213d8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc4223a0060, 0xc421b9c320, 0xc, 0xc, 0x3, 0x40df9e, 0xc4210b31a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc4223a0060, 0xc421b9c320, 0xc, 0xc, 0xc, 0xc, 0xc421839440, 0xc421824eb4) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc4223a0060, 0xc421b9c320, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421824e70, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421824e70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2875 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42119ec00, 0xc4225c9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc422300660) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc422300660, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42211a930, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42179a460) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422300840, 0x16e4b60, 0xc42179a4c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422300840, 0x16e4b60, 0xc42179a4c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4227461c0, 0x16e4b60, 0xc42179a4c0, 0xc42179a4c0, 0xc420cca6b8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4227461c0, 0xc42179a4c0, 0x2d, 0xc42249c8c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc420cca690, 0x1d72340, 0xc4227461c0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc420cca690, 0x1d72340, 0xc4227461c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc420cca690, 0x1d72340, 0xc4227461c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119ec00, 0xc420cca690) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1183 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421db8e70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 413 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4219ec630, 0xc4224ead10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4219ec630, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4213a7d20, 0xc4222ae780) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4213a7d20, 0xc4222ae780) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 414 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f6b8, 0x72, 0xc4224ecce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc422155518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc422155518, 0xc422596000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc422155500, 0xc422596000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc422155500, 0xc422596000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4213a7d20, 0xc422596000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc42287a1e0, 0xc422c0e140, 0xc, 0xc, 0x3, 0x40df9e, 0xc4219bd380) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc42287a1e0, 0xc422c0e140, 0xc, 0xc, 0xc, 0xc, 0xc421172a80, 0xc4219ec674) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc42287a1e0, 0xc422c0e140, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4219ec630, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4219ec630) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 415 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4219ec630) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 416 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4219ec630) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 417 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422b74960, 0xc4229e9700, 0x20, 0x20, 0x169d4e0, 0xc42052d220) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422016d20, 0x1d718c0, 0xc422b74960, 0x0, 0xc422b74900) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420286f00, 0xc4229e9950, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42287bf80, 0xc4228f2380, 0xc422016ba0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc420558a20, 0xc422b78028, 0x13, 0x1bda9a0, 0x4, 0xc4229e9d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc420558a20, 0xc422b78028, 0x13, 0xc4229e9d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc422635780, 0xc4222ae820, 0xc4228f5a58, 0x0, 0xc422561e00, 0xc4222defa0, 0x19758e0, 0xc4228f2380, 0x16, 0x169e020, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4222ae820, 0x1d72340, 0xc4226357c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc420155c80, 0xc4222ae820) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1565 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421502160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3416 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420286f00, 0x1be8427, 0xd, 0x1d76c80, 0xc422f83020, 0x1974aa0, 0xc422f83020, 0x16e76e0, 0xc4235ee090, 0x198, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Status).Leader(0xc42000ea68, 0xc422f83020, 0xc4235ee090, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/status_endpoint.go:46 +0xbd Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc4205226c0, 0xc4202107c0, 0x13, 0x1bda9a0, 0x4, 0xc422a54d48, 0x3, 0x3, 0x1734280, 0x18, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc4205226c0, 0xc4202107c0, 0x13, 0xc422a54d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc421fd2800, 0xc4214edc70, 0xc422f36b30, 0x0, 0xc420bf4880, 0xc422373e20, 0x1974aa0, 0xc422f83020, 0x16, 0x16e76e0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4214edc70, 0x1d72340, 0xc422500040, 0x0, 0xc42119f440) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119f440, 0xc4214edc70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2074 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420c80840, 0xc422a57d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420c80840, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc422b781e8, 0xc4210249b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc422b781e8, 0xc4210249b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2160 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421feabb0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 384 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421824e70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 382 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421824e70, 0xc422a60d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421824e70, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420211198, 0xc421f1c000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420211198, 0xc421f1c000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1906 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420994370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1808 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072ee98, 0x72, 0xc422a65ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc4224f1098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc4224f1098, 0xc42279d000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc4224f1080, 0xc42279d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc4224f1080, 0xc42279d000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4220dd238, 0xc42279d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc422691f20, 0xc422bce000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc422691f20, 0xc422bce000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc422691f20, 0xc422bce000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420994370, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420994370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2706 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072ec28, 0x72, 0xc422f26ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc4235e4298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc4235e4298, 0xc4235f6000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc4235e4280, 0xc4235f6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc4235e4280, 0xc4235f6000, 0x1000, 0x1000, 0xc420050dd8, 0x1, 0xc420050e08) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420218e00, 0xc4235f6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc423587620, 0xc422ea7c60, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc423587620, 0xc422ea7c60, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc423587620, 0xc422ea7c60, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc423490840, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc423490840) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 454 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f5e8, 0x72, 0xc423340ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc421bada18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc421bada18, 0xc421f1e000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc421bada00, 0xc421f1e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc421bada00, 0xc421f1e000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4220dc4b8, 0xc421f1e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc422707620, 0xc422c0f670, 0xc, 0xc, 0xc, 0x436544, 0xc423340e98) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc422707620, 0xc422c0f670, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc422707620, 0xc422c0f670, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420995ad0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420995ad0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 455 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420995ad0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 456 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420995ad0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1185 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421270540, 0xc422ba3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc422825860) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc422825860, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422086330, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42197b7a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4228258c0, 0x16e4b60, 0xc42197b820) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4228258c0, 0x16e4b60, 0xc42197b820, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421cae180, 0x16e4b60, 0xc42197b820, 0xc42197b820, 0xc421f1d568) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421cae180, 0xc42197b820, 0x2d, 0xc422d68060) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc421f1d540, 0x1d72340, 0xc421cae180, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc421f1d540, 0x1d72340, 0xc421cae180, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421f1d540, 0x1d72340, 0xc421cae180, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421270540, 0xc421f1d540) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1551 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4219bc120, 0xc42257b700, 0x20, 0x20, 0x169d4e0, 0xc42052d220) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4212da030, 0x1d718c0, 0xc4219bc120, 0x0, 0xc4219bc100) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420286f00, 0xc42257b950, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4216561e0, 0xc422ae5800, 0xc421027bc0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc420558a20, 0xc422b78870, 0x13, 0x1bda9a0, 0x4, 0xc42257bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc420558a20, 0xc422b78870, 0x13, 0xc42257bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc4215c7140, 0xc420cca690, 0xc420548a98, 0x0, 0xc422ae4d80, 0xc42057d920, 0x19758e0, 0xc422ae5800, 0x16, 0x169e020, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc420cca690, 0x1d72340, 0xc4215c7200, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421270b40, 0xc420cca690) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1752 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42119ee40, 0xc42173f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc422dcb740) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc422dcb740, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421d3ced0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421017e00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422dcb7a0, 0x16e4b60, 0xc421017e60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422dcb7a0, 0x16e4b60, 0xc421017e60, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4225ab8c0, 0x16e4b60, 0xc421017e60, 0xc421017e60, 0xc421ec1518) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4225ab8c0, 0xc421017e60, 0x2d, 0xc42348ada0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc421ec14f0, 0x1d72340, 0xc4225ab8c0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc421ec14f0, 0x1d72340, 0xc4225ab8c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421ec14f0, 0x1d72340, 0xc4225ab8c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119ee40, 0xc421ec14f0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2995 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4215e5760) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1853 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422b1ade0, 0xc4234fb208, 0x20, 0x20, 0xc421d43000, 0xc422565bf0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422565dd0, 0x1d718c0, 0xc422b1ade0, 0x0, 0xc422b1ad00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420286f00, 0xc4234fb440, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Job).List(0xc42000e9e8, 0xc422b1ad20, 0xc422565bf0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/job_endpoint.go:925 +0x2ea Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc42044dd40, 0xc42000f068, 0x13, 0x1bda9a0, 0x4, 0xc4234fb898, 0x3, 0x3, 0x192b7e0, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc42044dd40, 0xc42000f068, 0x13, 0xc42292a898, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc42042d580, 0xc42020b540, 0xc4225b1470, 0x0, 0xc420396a80, 0xc421252020, 0x1974e60, 0xc422b1ad20, 0x16, 0x169dd20, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc42020b540, 0x1d71b80, 0xc422350400, 0xc42044a400, 0xa) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420286f00, 0x1be08be, 0x8, 0x1974e60, 0xc422b1acc0, 0x169dd20, 0xc422565bc0, 0xc422b1ac01, 0xc422565bc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc4202a1c00, 0x1be08be, 0x8, 0x1974e60, 0xc422b1acc0, 0x169dd20, 0xc422565bc0, 0x0, 0xc42044a400) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).jobListRequest(0xc421248240, 0x1d6df80, 0xc422b1ac60, 0xc420b81000, 0x44774f, 0xc422afe900, 0xc422723340, 0x1d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:33 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest(0xc421248240, 0x1d6df80, 0xc422b1ac60, 0xc420b81000, 0xc422723370, 0xc42292ab58, 0x38, 0xc42292ab50) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:18 +0x1a6 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest-fm(0x1d6df80, 0xc422b1ac60, 0xc420b81000, 0xc420b81000, 0xc422e66580, 0x1d, 0xbed7c7f59e01310e) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:141 +0x48 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc422b1ac60, 0xc420b81000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.HandlerFunc.ServeHTTP(0xc4214f6300, 0x1d6df80, 0xc422b1ac60, 0xc420b81000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*ServeMux).ServeHTTP(0xc4214f4330, 0x1d6df80, 0xc422b1ac60, 0xc420b81000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc42185d420, 0xc420b81000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.HandlerFunc.ServeHTTP(0xc421265b60, 0x1d6fec0, 0xc42185d420, 0xc420b81000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.serverHandler.ServeHTTP(0xc42059dee0, 0x1d6fec0, 0xc42185d420, 0xc420b81000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*conn).serve(0xc422b31680, 0x1d71840, 0xc4223502c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by net/http.(*Server).Serve Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 619 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f448, 0x72, 0xc42292dce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc421964618, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc421964618, 0xc421153000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc421964600, 0xc421153000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc421964600, 0xc421153000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420210b50, 0xc421153000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc420e91800, 0xc421f67240, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc420e91800, 0xc421f67240, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc420e91800, 0xc421f67240, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421503290, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421503290) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1850 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422486300, 0xc4236f72c8, 0x20, 0x20, 0x5, 0x1bd9a98) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4222936e0, 0x1d718c0, 0xc422486300, 0x0, 0xc422486300) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420286f00, 0xc4236f7500, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Node).List(0xc4205587e0, 0xc4210715c0, 0xc422293080, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:1198 +0x27c Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc420558ae0, 0xc420556188, 0x13, 0x1bda9a0, 0x4, 0xc4229e78f8, 0x3, 0x3, 0x192ba60, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc420558ae0, 0xc420556188, 0x13, 0xc422a578f8, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc420554380, 0xc42020b540, 0xc4225f3c40, 0x0, 0xc420530600, 0xc4214e8020, 0x1975760, 0xc4210715c0, 0x16, 0x169e120, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc42020b540, 0x1d71b80, 0xc422e915c0, 0xc42044a400, 0xa) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420286f00, 0x1be205d, 0x9, 0x1975760, 0xc421071560, 0x169e120, 0xc422293050, 0xc421071501, 0xc422293050) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc4202a1c00, 0x1be205d, 0x9, 0x1975760, 0xc421071560, 0x169e120, 0xc422293050, 0xc4216dff80, 0x1e) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest(0xc421248240, 0x1d6df80, 0xc421071500, 0xc420ac8a00, 0xc4216dffb0, 0xc422a57b58, 0x38, 0xc422a57b50) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/node_endpoint.go:24 +0x1b3 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest-fm(0x1d6df80, 0xc421071500, 0xc420ac8a00, 0xc420ac8a00, 0xc4229657e0, 0x1e, 0xbed7c7f59dece84d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:145 +0x48 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc421071500, 0xc420ac8a00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.HandlerFunc.ServeHTTP(0xc4214f6360, 0x1d6df80, 0xc421071500, 0xc420ac8a00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*ServeMux).ServeHTTP(0xc4214f4330, 0x1d6df80, 0xc421071500, 0xc420ac8a00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc421273ea0, 0xc420ac8a00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.HandlerFunc.ServeHTTP(0xc421265b60, 0x1d6fec0, 0xc421273ea0, 0xc420ac8a00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.serverHandler.ServeHTTP(0xc42059dee0, 0x1d6fec0, 0xc421273ea0, 0xc420ac8a00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*conn).serve(0xc422b30f00, 0x1d71840, 0xc422e91480) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by net/http.(*Server).Serve Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2076 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420c80840) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2690 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421271740, 0xc4235d3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4235da1e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4235da1e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4215e6150, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42275fc20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4235da240, 0x16e4b60, 0xc42275fca0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4235da240, 0x16e4b60, 0xc42275fca0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422bb58c0, 0x16e4b60, 0xc42275fca0, 0xc42275fca0, 0xc421255ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422bb58c0, 0xc42275fca0, 0x2d, 0xc42354dc20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc421255cc0, 0x1d72340, 0xc422bb58c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc421255cc0, 0x1d72340, 0xc422bb58c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421255cc0, 0x1d72340, 0xc422bb58c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421271740, 0xc421255cc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 621 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421503290) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 622 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421163920, 0xc422649700, 0x20, 0x20, 0x169d4e0, 0xc42052d220) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421d81d10, 0x1d718c0, 0xc421163920, 0x0, 0xc421163900) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420286f00, 0xc422649950, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4211631a0, 0xc422606600, 0xc421d81b00, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc420558a20, 0xc420210e58, 0x13, 0x1bda9a0, 0x4, 0xc423342d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc420558a20, 0xc420210e58, 0x13, 0xc423342d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc422027280, 0xc421068460, 0xc4210507c8, 0x0, 0xc422606180, 0xc421cce960, 0x19758e0, 0xc422606600, 0x16, 0x169e020, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421068460, 0x1d72340, 0xc422027440, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119ea80, 0xc421068460) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 635 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4221e6180, 0xc422899000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc421250300) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc421250300, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421f576e0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421c93ba0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421250480, 0x16e4b60, 0xc421c93c00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421250480, 0x16e4b60, 0xc421c93c00, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421c85b40, 0x16e4b60, 0xc421c93c00, 0xc421c93c00, 0xc421068488) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421c85b40, 0xc421c93c00, 0x2d, 0xc4225fc920) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc421068460, 0x1d72340, 0xc421c85b40, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc421068460, 0x1d72340, 0xc421c85b40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421068460, 0x1d72340, 0xc421c85b40, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc4221e6180, 0xc421068460) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2161 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421feabb0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1820 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f038, 0x72, 0xc422c97658) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc42109b198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc42109b198, 0xc422292d00, 0x1, 0x1) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc42109b180, 0xc422292d01, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc42109b180, 0xc422292d01, 0x1, 0x1, 0x20002, 0xc422c977b4, 0xc422c977b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420557ba0, 0xc422292d01, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*connReader).backgroundRead(0xc422292cf0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 620 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421503290) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 618 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421503290, 0xc423341d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421503290, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420210b50, 0xc4210683c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420210b50, 0xc4210683c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1182 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f2a8, 0x72, 0xc422a61ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc421d36518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc421d36518, 0xc422ba2000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc421d36500, 0xc422ba2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc421d36500, 0xc422ba2000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420557ef0, 0xc422ba2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc422691560, 0xc421b9d020, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc422691560, 0xc421b9d020, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc422691560, 0xc421b9d020, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421db8e70, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421db8e70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2075 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072edc8, 0x72, 0xc422927ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc4219b4c18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc4219b4c18, 0xc421dd9000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc4219b4c00, 0xc421dd9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc4219b4c00, 0xc421dd9000, 0x1000, 0x1000, 0xc422aec5d8, 0x1, 0xc422aec608) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc422b781e8, 0xc421dd9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc422b20180, 0xc421eab980, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc422b20180, 0xc421eab980, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc422b20180, 0xc421eab980, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420c80840, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420c80840) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1562 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421502160, 0xc423343d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421502160, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4220dc850, 0xc421255bd0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4220dc850, 0xc421255bd0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1564 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421502160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2159 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072ecf8, 0x72, 0xc422aea4e8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc422ae5f98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc422ae5f98, 0xc422600000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc422ae5f80, 0xc422600000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc422ae5f80, 0xc422600000, 0x1000, 0x1000, 0xc422aea5d8, 0x1, 0xc422aea608) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4220dce70, 0xc422600000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc42107b6e0, 0xc4206ff210, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc42107b6e0, 0xc4206ff210, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc42107b6e0, 0xc4206ff210, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421feabb0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421feabb0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3466 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc42056c580, 0xc4221e6780, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc42056c580, 0xc4221e6780) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42056c580, 0xc42288df20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2735 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072eb58, 0x72, 0xc422f25ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc4227ac218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc4227ac218, 0xc421b75000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc4227ac200, 0xc421b75000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc4227ac200, 0xc421b75000, 0x1000, 0x1000, 0xc422ae6dd8, 0x1, 0xc422ae6e08) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc42000e728, 0xc421b75000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc421950e40, 0xc422a4c160, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc421950e40, 0xc422a4c160, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc421950e40, 0xc422a4c160, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420994210, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420994210) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2897 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072e9b8, 0x72, 0xc420fa4ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc4215a0118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc4215a0118, 0xc421976000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc4215a0100, 0xc421976000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc4215a0100, 0xc421976000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420218c08, 0xc421976000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc421ea40c0, 0xc422214ab4, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc421ea40c0, 0xc422214ab4, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc421ea40c0, 0xc422214ab4, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4215e5760, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4215e5760) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1566 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42044c600, 0xc422581700, 0x20, 0x20, 0x169d4e0, 0xc42052d220) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420c34d50, 0x1d718c0, 0xc42044c600, 0x0, 0xc42044c600) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420286f00, 0xc422581950, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42105a0c0, 0xc420770700, 0xc420c34a20, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc420558a20, 0xc4220dcb58, 0x13, 0x1bda9a0, 0x4, 0xc422581d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc420558a20, 0xc4220dcb58, 0x13, 0xc422581d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc42116a1c0, 0xc421255cc0, 0xc421b8ed08, 0x0, 0xc4220bbb00, 0xc4203c0f20, 0x19758e0, 0xc420770700, 0x16, 0x169e020, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421255cc0, 0x1d72340, 0xc42116a280, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119ed80, 0xc421255cc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3511 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc42124a000, 0x1d71840, 0xc422501040) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1563 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f1d8, 0x72, 0xc42335ace8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc42110d218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc42110d218, 0xc421b4f000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc42110d200, 0xc421b4f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc42110d200, 0xc421b4f000, 0x1000, 0x1000, 0xc42335add8, 0x1, 0xc42335ae08) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4220dc850, 0xc421b4f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc4213c4b40, 0xc422119480, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc4213c4b40, 0xc422119480, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc4213c4b40, 0xc422119480, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421502160, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421502160) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1809 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420994370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3082 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420c80c60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 998 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420c81ef0, 0xc423342d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420c81ef0, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420557bd0, 0xc421627770) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420557bd0, 0xc421627770) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1803 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072ef68, 0x72, 0xc42055be58) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc42109b398, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc42109b398, 0xc422565900, 0x1, 0x1) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc42109b380, 0xc4225659c1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc42109b380, 0xc4225659c1, 0x1, 0x1, 0x20002, 0xc42055bfb4, 0xc42055bfb0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420557d00, 0xc4225659c1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.(*connReader).backgroundRead(0xc4225659b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2077 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420c80840) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1015 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f378, 0x72, 0xc42335dce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc421bac998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc421bac998, 0xc422901000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc421bac980, 0xc422901000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc421bac980, 0xc422901000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420557bd0, 0xc422901000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc422b20960, 0xc4222a28b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc422b20960, 0xc4222a28b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc422b20960, 0xc4222a28b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420c81ef0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420c81ef0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3081 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8ad0285a0, 0x72, 0xc422f21ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc420ebd898, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc420ebd898, 0xc420c92000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc420ebd880, 0xc420c92000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc420ebd880, 0xc420c92000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4213a78c8, 0xc420c92000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc4212506c0, 0xc42071e8a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc4212506c0, 0xc42071e8a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc4212506c0, 0xc42071e8a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420c80c60, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420c80c60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3508 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422501080, 0xc4214db770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422760ed0, 0x1d71840, 0xc422501080, 0x0, 0x1849b5) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42001a0c0, 0xc4214db960, 0x1849b5, 0x1d71840, 0xc422501080, 0x552bde, 0xc4214db950, 0xc4214db970, 0xf33ba5, 0xc422f65788) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc422f65730, 0x1d71840, 0xc422501080, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc422f65730) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1547 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420240370, 0xc42335bd10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420240370, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc422b78568, 0xc420cca5a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc422b78568, 0xc420cca5a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1017 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420c81ef0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2657 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc423490840, 0xc4224e6d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc423490840, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420218e00, 0xc422a9f950) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420218e00, 0xc422a9f950) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1548 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c072f108, 0x72, 0xc422929ce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc420e88198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc420e88198, 0xc421e93000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc420e88180, 0xc421e93000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc420e88180, 0xc421e93000, 0x1000, 0x1000, 0xc422929dd8, 0x1, 0xc422929e08) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc422b78568, 0xc421e93000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc42031ad80, 0xc420a63350, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc42031ad80, 0xc420a63350, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc42031ad80, 0xc420a63350, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420240370, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420240370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1181 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421db8e70, 0xc42335ed10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421db8e70, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420557ef0, 0xc421f1d450) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420557ef0, 0xc421f1d450) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3507 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422501000, 0xc422a62ce8, 0x20, 0x20, 0x18, 0xc422e97aa0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42274d2c0, 0x1d71840, 0xc422501000, 0xc422e97aa0, 0x184987) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42001a0c0, 0xc4227a5ed0, 0x184987, 0x1d71840, 0xc422501000, 0xc420cdc360, 0xc4227605d0, 0xc420b8fce0, 0x24, 0x2bba400) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420cdc300, 0x1d71840, 0xc422501000, 0x184987, 0xc421c639c0, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420cdc300, 0x1d71840, 0xc422501000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1018 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4212703c0, 0xc4211e8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc422310c60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc422310c60, 0xc4229e7a70, 0x4120a6, 0x7fe8c080efe8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421720210, 0x3c) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42060d400) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422310cc0, 0x16e4b60, 0xc42060d460) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422310cc0, 0x16e4b60, 0xc42060d460, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42191cd40, 0x16e4b60, 0xc42060d460, 0xc42060d460, 0xc4210520c8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42191cd40, 0xc42060d460, 0x2d, 0xc421f23180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc4210520a0, 0x1d72340, 0xc42191cd40, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc4210520a0, 0x1d72340, 0xc42191cd40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4210520a0, 0x1d72340, 0xc42191cd40, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc4212703c0, 0xc4210520a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3080 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420c80c60, 0xc422a50d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420c80c60, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4213a78c8, 0xc4214b1360) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4213a78c8, 0xc4214b1360) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1549 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420240370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1907 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42119efc0, 0xc4227f5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4230d5080) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4230d5080, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422e8ad20, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4226a7b00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4230d50e0, 0x16e4b60, 0xc4226a7b60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4230d50e0, 0x16e4b60, 0xc4226a7b60, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421eafbc0, 0x16e4b60, 0xc4226a7b60, 0xc4226a7b60, 0xc422dffec8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421eafbc0, 0xc4226a7b60, 0x2d, 0xc422d5edc0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc422dffea0, 0x1d72340, 0xc421eafbc0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc422dffea0, 0x1d72340, 0xc421eafbc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc422dffea0, 0x1d72340, 0xc421eafbc0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119efc0, 0xc422dffea0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2707 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc423490840) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3084 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421271380, 0xc420c93000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4210b26c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4210b26c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42111af30, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420646360) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210b2840, 0x16e4b60, 0xc4206463c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210b2840, 0x16e4b60, 0xc4206463c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421715a80, 0x16e4b60, 0xc4206463c0, 0xc4206463c0, 0xc4214b1428) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421715a80, 0xc4206463c0, 0x2d, 0xc421a8a120) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc4214b1400, 0x1d72340, 0xc421715a80, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc4214b1400, 0x1d72340, 0xc421715a80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4214b1400, 0x1d72340, 0xc421715a80, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421271380, 0xc4214b1400) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 1550 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420240370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2078 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421270c00, 0xc420f1a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4226c2c60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4226c2c60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421d3c990, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421a790e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4226c2cc0, 0x16e4b60, 0xc421a79140) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4226c2cc0, 0x16e4b60, 0xc421a79140, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42222e080, 0x16e4b60, 0xc421a79140, 0xc421a79140, 0xc421024a78) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42222e080, 0xc421a79140, 0x2d, 0xc42270d600) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc421024a50, 0x1d72340, 0xc42222e080, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc421024a50, 0x1d72340, 0xc42222e080, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc421024a50, 0x1d72340, 0xc42222e080, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421270c00, 0xc421024a50) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2709 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42119f140, 0xc4235f7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc422f1f920) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc422f1f920, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422679e60, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42230f000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422f1f980, 0x16e4b60, 0xc42230f060) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422f1f980, 0x16e4b60, 0xc42230f060, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422bd9200, 0x16e4b60, 0xc42230f060, 0xc42230f060, 0xc422a9fa18) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422bd9200, 0xc42230f060, 0x2d, 0xc4230d7ee0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc422a9f9f0, 0x1d72340, 0xc422bd9200, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc422a9f9f0, 0x1d72340, 0xc422bd9200, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc422a9f9f0, 0x1d72340, 0xc422bd9200, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc42119f140, 0xc422a9f9f0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2736 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420994210) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3510 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422501100) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2994 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4215e5760) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2737 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420994210) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 2786 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421270000, 0xc420e51000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc4223c79e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc4223c79e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422511bf0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42275f1e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4223c7a40, 0x16e4b60, 0xc42275f240) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4223c7a40, 0x16e4b60, 0xc42275f240, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422b685c0, 0x16e4b60, 0xc42275f240, 0xc42275f240, 0xc4224509d8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422b685c0, 0xc42275f240, 0x2d, 0xc4227fba60) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequestHeader(0xc4224509b0, 0x1d72340, 0xc422b685c0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020b220, 0x8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).readRequest(0xc4224509b0, 0x1d72340, 0xc422b685c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4224509b0, 0x1d72340, 0xc422b685c0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421270000, 0xc4224509b0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3424 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420152630) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3509 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422501040, 0xc422f33a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42274d440, 0x1d71840, 0xc422501040, 0xc42274d560, 0x1833a4) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42001a0c0, 0xc422f33c68, 0x1833a4, 0x1d71840, 0xc422501040, 0x552bde, 0xc42020b2c0, 0x2, 0xc422a43ac0, 0x40) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc4225010c0, 0x1833a4, 0x3b, 0xc42125bde8, 0x1, 0x1) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc4225010c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3467 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/http.setRequestCancel.func3(0x0, 0xc4221ffce0, 0xc4237b6410, 0xc422f6b71c, 0xc421b26000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/client.go:323 +0xfa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by net/http.setRequestCancel Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/http/client.go:322 +0x2ac Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3422 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc42020b4f0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc420286f00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3506 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc422760660) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3461 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420286f00, 0x1be6698, 0xc, 0x1d76b80, 0xc422faa4e0, 0x19746e0, 0xc422faa4e0, 0x181f280, 0xc422cf8c00, 0xc42044d6e0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc42000e9e0, 0xc422faa4e0, 0xc422cf8c00, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc42044d6e0, 0xc420210638, 0x13, 0x1bda9a0, 0x4, 0xc422f23d48, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc42044d6e0, 0xc420210638, 0x13, 0xc422f23d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc421fd2180, 0xc4214edc70, 0xc4230f29d0, 0x0, 0xc421021380, 0xc422373840, 0x19746e0, 0xc422faa4e0, 0x16, 0x181f280, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc4214edc70, 0x1d72340, 0xc4222d5e00, 0x0, 0xc4212715c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc4212715c0, 0xc4214edc70) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3465 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8ad028400, 0x72, 0xc42264f858) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc42377e418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc42377e418, 0xc4237a8000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc42377e400, 0xc4237a8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc42377e400, 0xc4237a8000, 0x1000, 0x1000, 0xc420040000, 0xc4213b8118, 0xc4213b8120) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc422b79a78, 0xc4237a8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).fill(0xc422f835c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).ReadByte(0xc422f835c0, 0xca1, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422317bf0, 0xc42377e418) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422ef0120) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422f83620, 0x16e76e0, 0xc4231e5a50) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422f83620, 0x16e76e0, 0xc4231e5a50, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.decodeResponse(0xc422500380, 0x182ba60, 0xc42354ce60, 0xc423772de0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:608 +0x5e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).genericRPC(0xc4200cc2c0, 0xc4205503f0, 0x24, 0xc42024d2c0, 0x11, 0x2570345523772d00, 0x182ba00, 0xc423772de0, 0x182ba60, 0xc42354ce60, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:406 +0xf9 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).AppendEntries(0xc4200cc2c0, 0xc4205503f0, 0x24, 0xc42024d2c0, 0x11, 0xc423772de0, 0xc42354ce60, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:379 +0x97 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc42056c580, 0xc4221e66c0, 0x186327, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:186 +0x189 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc42056c580, 0xc4221e66c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:132 +0x3f2 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42056c580, 0xc42288df00) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3425 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420152630) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3421 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420718aa0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3474 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200766e0, 0xc42122ad10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200766e0, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420219d38, 0xc42376a000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc420219d38, 0xc42376a000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3475 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8c08032d0, 0x72, 0xc42358ace8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc422f3b718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc422f3b718, 0xc422fcd000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc422f3b700, 0xc422fcd000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc422f3b700, 0xc422fcd000, 0x1000, 0x1000, 0xc422aeb5d8, 0x1, 0xc422aeb608) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc420219d38, 0xc422fcd000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc422fab500, 0xc4230f2ad0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc422fab500, 0xc4230f2ad0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc422fab500, 0xc4230f2ad0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200766e0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200766e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3476 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200766e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3477 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200766e0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3478 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420286f00, 0x1bef5ff, 0x11, 0x1d77180, 0xc42376f8c0, 0x1975b20, 0xc42376f8c0, 0x169e1e0, 0xc4227e3800, 0xc42238a1b0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Node).UpdateStatus(0xc42376f2c0, 0xc42376f8c0, 0xc4227e3800, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:321 +0xa9 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.call(0xc420558cc0, 0xc420556cf8, 0x13, 0x1bda9a0, 0x4, 0xc421e17d48, 0x3, 0x3, 0x1a99fe0, 0x19, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: reflect.Value.Call(0xc420558cc0, 0xc420556cf8, 0x13, 0xc421e17d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*service).call(0xc422cf95c0, 0xc42376a0a0, 0xc422f8edf0, 0x0, 0xc4227e3700, 0xc422e12cc0, 0x1975b20, 0xc42376f8c0, 0x16, 0x169e1e0, ...) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net/rpc.(*Server).ServeRequest(0xc42376a0a0, 0x1d72340, 0xc422cf9600, 0x0, 0xc421271680) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d0e0, 0xc421271680, 0xc42376a0a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3468 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc421c63a80, 0xc422f80d28, 0x20, 0x20, 0xc42017e400, 0x1bd9a98) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42278a150, 0x1d71840, 0xc421c63a80, 0xc422e97c40, 0x1849b5) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42001a0c0, 0xc422baff20, 0x1849b5, 0x1d71840, 0xc421c63a80, 0xc423772a80, 0x30003, 0xc422aeb79e, 0xc422aeb798, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc4223d2d20, 0x1849b5, 0x1, 0x0, 0x0, 0x0, 0xc421839680, 0x71ce7c) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc4223d2d20, 0x1849b5, 0xc4237bdaa0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3483 [runnable]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldEnqueue(0xc421b4d080, 0x1a1c020) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6594 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc420286f00, 0xc42001a001, 0xc42001a0c0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:298 +0x117 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc420286f00, 0xc4218397a0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc420286f00, 0xc4218397a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc421258010, 0xc420286f00, 0xc4218397a0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3484 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc42056c580, 0xc4221e66c0, 0xc421839800) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42056c580, 0xc422e96fa0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3418 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc422500480) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3486 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc42056c580, 0xc4221e6780, 0xc421839980) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42056c580, 0xc422e97020) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3487 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200768f0, 0xc422a52d10, 0x30, 0xf9866a) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200768f0, 0x1c61988, 0xc420286f00, 0x1d71840, 0xc421248180) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4220dd820, 0xc4237a4370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420286f00, 0x1d71840, 0xc421248180, 0x1d7d320, 0xc4220dd820, 0xc4237a4370) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3488 [IO wait]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.runtime_pollWait(0x7fe8ad028330, 0x72, 0xc42292cce8) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).wait(0xc42377e498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*pollDesc).waitRead(0xc42377e498, 0xc4237d0000, 0x1000, 0x1000) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: internal/poll.(*FD).Read(0xc42377e480, 0xc4237d0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*netFD).Read(0xc42377e480, 0xc4237d0000, 0x1000, 0x1000, 0xc42292cdd8, 0x0, 0xc42292ce08) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: net.(*conn).Read(0xc4220dd820, 0xc4237d0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: bufio.(*Reader).Read(0xc4237732c0, 0xc422f6b740, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadAtLeast(0x1d59aa0, 0xc4237732c0, 0xc422f6b740, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: io.ReadFull(0x1d59aa0, 0xc4237732c0, 0xc422f6b740, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200768f0, 0x0, 0x0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200768f0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3489 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200768f0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3490 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200768f0) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: goroutine 3493 [select]: Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc4223d2d20) Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:28:23 ip-172-24-20-236 nomad.sh[25799]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:28:24 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:28:24 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:28:24 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: ==> Starting Nomad agent... Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: ==> Nomad agent configuration: Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: Client: false Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: Log Level: INFO Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: Region: amrv01 (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: Server: true Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: Version: 0.8.4 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.472238 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: Attempting re-join to previously known node: ip-172-25-5-80.amv01: 172.25.5.80:4648 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.472960 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [WARN] memberlist: Refuting an alive message Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828141 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828161 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828174 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828185 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828196 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828207 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828242 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828417 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828433 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828445 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828456 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828467 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828588 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828602 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828614 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828657 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828888 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24 [INFO] serf: Re-joined to previously known node: ip-172-25-5-80.amv01: 172.25.5.80:4648 Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828903 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828925 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:28:24 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:24.828937 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:25 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:25 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:28 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:28.533090 [ERR] worker: failed to ack evaluation '47dad366-3311-35ab-2787-d83785faeada': rpc error: EOF Aug 23 15:28:28 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:28.542525 [ERR] worker: failed to update evaluation : rpc error: stream closed Aug 23 15:28:28 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:28.542529 [ERR] worker: failed to update evaluation : rpc error: stream closed Aug 23 15:28:28 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:28.543545 [ERR] worker: failed to nack evaluation '47dad366-3311-35ab-2787-d83785faeada': rpc error: failed to get conn: dial tcp 172.24.10.89:4647: connect: connection refused Aug 23 15:28:28 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:28.543565 [ERR] worker: failed to update evaluation : rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:28:28 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:28.543574 [ERR] worker: failed to nack evaluation '47dad366-3311-35ab-2787-d83785faeada': rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:28:28 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:28.544507 [ERR] worker: failed to nack evaluation '47dad366-3311-35ab-2787-d83785faeada': rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:29 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:29 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 46 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:29 [INFO] raft: Election won. Tally: 2 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:29 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:29 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:29 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:29.857770 [INFO] nomad: cluster leadership acquired Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: 2018/08/23 15:28:29 [INFO] raft: pipelining replication to peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: fatal error: concurrent map writes Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 272 [running]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc421400d08 sp=0xc421400ce8 pc=0x42d161 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: runtime.mapdelete(0x1844be0, 0xc42051a1e0, 0xc421400e20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc421400d68 sp=0xc421400d08 pc=0x40a761 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc42027f9e0, 0x1d6dd40, 0xc42000ffe8, 0x0, 0xed3109f6d) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc421400e50 sp=0xc421400d68 pc=0xf2164c Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc420420000, 0x1d71840, 0xc422bf5400) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc421400fc8 sp=0xc421400e50 pc=0xf56bb7 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: runtime.goexit() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc421400fd0 sp=0xc421400fc8 pc=0x45c711 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 1 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42021f730, 0xc42041c640) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42021f730, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4203e2640, 0xc4203e2640, 0xc420212960, 0x37) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204d7f68) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc42009e058) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: main.main() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 5 [syscall]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: os/signal.signal_recv(0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: os/signal.loop() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by os/signal.init.0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 40 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc420316300) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 144 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a2d0, 0x72, 0xc420564ce8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc42141a018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc42141a018, 0xc42131b000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc42141a000, 0xc42131b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc42141a000, 0xc42131b000, 0x1000, 0x1000, 0xc42055ddd8, 0x1, 0xc42055de08) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc42000e598, 0xc42131b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc4213c4060, 0xc42116e7b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadAtLeast(0x1d59aa0, 0xc4213c4060, 0xc42116e7b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadFull(0x1d59aa0, 0xc4213c4060, 0xc42116e7b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4201ee210, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4201ee210) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 39 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420512300) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 38 [sleep]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: time.Sleep(0x3b9aca00) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420510000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 37 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc420427d00) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 55 [select, locked to thread]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: runtime.selectgo(0xc4204b5750, 0xc420146480) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: runtime.ensureSigM.func1() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: runtime.goexit() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 57 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4202b0fc0, 0x1d71840, 0xc420427d40) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 42 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc42040ed00) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc42051a630, 0xc42015cce0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 143 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4201ee210, 0xc420615d10, 0x30, 0xf9866a) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4201ee210, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc42000e598, 0xc4213fa000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc42000e598, 0xc4213fa000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 227 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4205120c0, 0xc420dfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc420f5a480) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc420f5a480, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42109f6b0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4213f00a0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420f5a4e0, 0x16e4b60, 0xc4213f0120) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420f5a4e0, 0x16e4b60, 0xc4213f0120, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421340440, 0x16e4b60, 0xc4213f0120, 0xc4213f0120, 0xc4213fa1b8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421340440, 0xc4213f0120, 0x2d, 0xc420794a20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequestHeader(0xc4213fa190, 0x1d72340, 0xc421340440, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420432a00, 0x8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequest(0xc4213fa190, 0x1d72340, 0xc421340440, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc4213fa190, 0x1d72340, 0xc421340440, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc4205120c0, 0xc4213fa190) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 66 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3af00, 0x72, 0xc4205699a8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc42021a398, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc42021a398, 0xc420586000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc42021a380, 0xc420586000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc42021a380, 0xc420586000, 0x1000, 0x1000, 0x458600, 0xc420eacf00, 0x4) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc4204a4090, 0xc420586000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.(*persistConn).Read(0xc4200dab40, 0xc420586000, 0x1000, 0x1000, 0xc420569b98, 0x405675, 0xc42009e720) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc420086960) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Peek(0xc420086960, 0x1, 0x0, 0x0, 0x1, 0xc420d7e720, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.(*persistConn).readLoop(0xc4200dab40) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by net/http.(*Transport).dialConn Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 67 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.(*persistConn).writeLoop(0xc4200dab40) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by net/http.(*Transport).dialConn Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 43 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4202ff400, 0x1c636a0, 0xc420510160, 0x1d71840, 0xc420599bc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc420510160) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 147 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200c22c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc4202052d0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 226 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4201ee210) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 124 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42031e840, 0xc421403d10, 0x30, 0xf9866a) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42031e840, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc420207588, 0xc420ce0dc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc420207588, 0xc420ce0dc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 183 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4201ee0b0, 0xc420562d10, 0x30, 0xf9866a) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4201ee0b0, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc4204a4098, 0xc420cd81e0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:29 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc4204a4098, 0xc420cd81e0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 180 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3aa20, 0x72, 0xc4214029a8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc420a66d18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc420a66d18, 0xc420ccf000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc420a66d00, 0xc420ccf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc420a66d00, 0xc420ccf000, 0x1000, 0x1000, 0x458600, 0xc42048c600, 0x4) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc4204a4030, 0xc420ccf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.(*persistConn).Read(0xc421370ea0, 0xc420ccf000, 0x1000, 0x1000, 0xc421402b98, 0x405675, 0xc420e9eea0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc4213e04e0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Peek(0xc4213e04e0, 0x1, 0x0, 0x0, 0x1, 0xc420e80480, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.(*persistConn).readLoop(0xc421370ea0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by net/http.(*Transport).dialConn Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 146 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200c22c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200c22c0) Aug 23 15:28:29 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200c22c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc4202052c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 145 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4201ee210) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 241 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc422c45a80) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 137 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420318540, 0xc420595000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc4213fc8a0) Aug 23 15:28:29 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc4213fc8a0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420cd24b0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420cde720) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4213fc900, 0x16e4b60, 0xc420cde780) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4213fc900, 0x16e4b60, 0xc420cde780, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420ca8180, 0x16e4b60, 0xc420cde780, 0xc420cde780, 0xc420cd82a8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420ca8180, 0xc420cde780, 0x2d, 0xc4209e24c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequestHeader(0xc420cd8280, 0x1d72340, 0xc420ca8180, 0x30, 0x1, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequest(0xc420cd8280, 0x1d72340, 0xc420ca8180, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc420cd8280, 0x1d72340, 0xc420ca8180, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc420318540, 0xc420cd8280) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 148 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200c22c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc4202052e0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 149 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420cb6870) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 150 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4209cc270) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 151 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3ad60, 0x72, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc420239898, 0x72, 0xc420083a00, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc420239898, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Accept(0xc420239880, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).accept(0xc420239880, 0xc420210020, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*TCPListener).accept(0xc42000eb20, 0x458e40, 0xc4204b2790, 0xc4204b2798) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*TCPListener).AcceptTCP(0xc42000eb20, 0x1c63658, 0xc4205143c0, 0xc420210020) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4205143c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 152 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3ac90, 0x72, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc420239a98, 0x72, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc420239a98, 0xc423390000, 0x10000, 0x10000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).ReadFrom(0xc420239a80, 0xc423390000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).readFrom(0xc420239a80, 0xc423390000, 0x10000, 0x10000, 0x96, 0x96, 0x1d62b40, 0x412539, 0xc420c6ee40) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*UDPConn).readFrom(0xc42000eb28, 0xc423390000, 0x10000, 0x10000, 0x101, 0xc420c6ee18, 0xc420c6ee90, 0x443cf7) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*UDPConn).ReadFrom(0xc42000eb28, 0xc423390000, 0x10000, 0x10000, 0x10000, 0x10000, 0xdcb6, 0x13a0ae200, 0x2b96de0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4205143c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 153 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4205143c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 154 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4205143c0, 0x12a05f200, 0xc420ce3800, 0xc420fbe3c0, 0xc420205970) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 155 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4205143c0, 0xc420fbe3c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 156 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4205143c0, 0x1dcd6500, 0xc420ce3860, 0xc420fbe3c0, 0xc420205980) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 157 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc420434d80) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 158 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc420434d80) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 159 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420434d80, 0x1bdd09a, 0x6, 0xc420cb69c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 160 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420434d80, 0x1bdba6c, 0x5, 0xc420cb69f0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 161 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420434d80, 0x1bdbc6a, 0x5, 0xc420cb6a20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 256 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a950, 0x72, 0xc4230d8ce8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc4207d7818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc4207d7818, 0xc4210b4000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc4207d7800, 0xc4210b4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc4207d7800, 0xc4210b4000, 0x1000, 0x1000, 0x48, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc420210980, 0xc4210b4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc421485b00, 0xc42267d460, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadAtLeast(0x1d59aa0, 0xc421485b00, 0xc42267d460, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadFull(0x1d59aa0, 0xc421485b00, 0xc42267d460, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4222ec160, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4222ec160) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 163 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4201f0e40, 0xc420e57000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc4209d23c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc4209d23c0, 0xc420042500, 0xc42102a258, 0xc42102a260) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4227971d0, 0xc420bb5410) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421d6d560) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4209d2420, 0x16e4ba0, 0xc4227974d0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4209d2420, 0x16e4ba0, 0xc4227974d0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422ce3cc0, 0x16e4ba0, 0xc4227974d0, 0x1, 0xc4227974d0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc422ce3cc0, 0xc4227974d0, 0x19746e0, 0xc4211a1320) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc422ce3cc0, 0x1be6698, 0xc, 0x19746e0, 0xc4211a1320, 0x181f280, 0xc422c8e8c0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc420316300, 0xc42003fa51, 0x6, 0x1d62b00, 0xc420ba0360, 0x1, 0x1be6698, 0xc, 0x19746e0, 0xc4211a1320, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc4203e8000, 0xc4204ca1c0, 0x1be6698, 0xc, 0x19746e0, 0xc4211a1320, 0x181f280, 0xc422c8e8c0, 0x1d59ee0, 0x1947b20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc4203e8000, 0x1be6698, 0xc, 0x1d76b80, 0xc4211a1320, 0x19746e0, 0xc4211a1320, 0x181f280, 0xc422c8e8c0, 0xc420316de0, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc420206170, 0xc4211a1320, 0xc422c8e8c0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.call(0xc420316de0, 0xc420206478, 0x13, 0x1bda9a0, 0x4, 0xc420bb5d18, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.Call(0xc420316de0, 0xc420206478, 0x13, 0xc420bb5d18, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*service).call(0xc4202b1100, 0xc4202feeb0, 0xc422b83088, 0x0, 0xc420143100, 0xc4213cef00, 0x19746e0, 0xc4211a1320, 0x16, 0x181f280, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc4202feeb0, 0x1d71b80, 0xc422c8e840, 0x5b7ea86c, 0xc439a3816a) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc4203e8000, 0x1be6698, 0xc, 0x19746e0, 0xc421f19e00, 0x181f280, 0xc422ce28c0, 0x1d59ee0, 0xc42140eda0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ce38c0, 0x1dcd6500, 0x24, 0xc422fdf170, 0x24, 0xc420626b00, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:158 +0x1bb Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ce38c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 164 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421202840, 0xc422bf7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc4209d2480) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc4209d2480, 0xc420042500, 0xc420f6b1d8, 0xc420f6b1e0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422797500, 0xc420f0f410) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421d6d660) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4209d24e0, 0x16e4ba0, 0xc422797860) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4209d24e0, 0x16e4ba0, 0xc422797860, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422ce3dc0, 0x16e4ba0, 0xc422797860, 0x1, 0xc422797860) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc422ce3dc0, 0xc422797860, 0x19746e0, 0xc4209d2120) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc422ce3dc0, 0x1be6698, 0xc, 0x19746e0, 0xc4209d2120, 0x181f280, 0xc422ce3c80, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc420316300, 0xc42003fa51, 0x6, 0x1d62b00, 0xc420ba0360, 0x1, 0x1be6698, 0xc, 0x19746e0, 0xc4209d2120, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc4203e8000, 0xc4204ca1c0, 0x1be6698, 0xc, 0x19746e0, 0xc4209d2120, 0x181f280, 0xc422ce3c80, 0x1d59ee0, 0x1947b20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc4203e8000, 0x1be6698, 0xc, 0x1d76b80, 0xc4209d2120, 0x19746e0, 0xc4209d2120, 0x181f280, 0xc422ce3c80, 0xc420316de0, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc420206170, 0xc4209d2120, 0xc422ce3c80, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.call(0xc420316de0, 0xc420206478, 0x13, 0x1bda9a0, 0x4, 0xc420f0fd18, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.Call(0xc420316de0, 0xc420206478, 0x13, 0xc420f0fd18, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*service).call(0xc4202b1100, 0xc4202feeb0, 0xc42267d488, 0x0, 0xc420143100, 0xc420cd4180, 0x19746e0, 0xc4209d2120, 0x16, 0x181f280, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc4202feeb0, 0x1d71b80, 0xc422ce3c00, 0x5b7ea86c, 0xc439a2fc1c) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc4203e8000, 0x1be6698, 0xc, 0x19746e0, 0xc42175b3e0, 0x181f280, 0xc4229b8980, 0x1d59ee0, 0xc420627050) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ce3920, 0x1dcd6500, 0x24, 0xc422fdef90, 0x24, 0xc420068d00, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:158 +0x1bb Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ce3920) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 165 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421202780, 0xc4210b5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc421485f20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc421485f20, 0xc420042500, 0xc4216d0c58, 0xc4216d0c60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422796e40, 0xc420fd9410) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421d6d420) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421485f80, 0x16e4ba0, 0xc422797110) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421485f80, 0x16e4ba0, 0xc422797110, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422ce3b40, 0x16e4ba0, 0xc422797110, 0x1, 0xc422797110) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc422ce3b40, 0xc422797110, 0x19746e0, 0xc4211a1200) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc422ce3b40, 0x1be6698, 0xc, 0x19746e0, 0xc4211a1200, 0x181f280, 0xc422c8e800, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc420316300, 0xc42003fa51, 0x6, 0x1d62b00, 0xc420ba0360, 0x1, 0x1be6698, 0xc, 0x19746e0, 0xc4211a1200, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc4203e8000, 0xc4204ca1c0, 0x1be6698, 0xc, 0x19746e0, 0xc4211a1200, 0x181f280, 0xc422c8e800, 0x1d59ee0, 0x1947b20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc4203e8000, 0x1be6698, 0xc, 0x1d76b80, 0xc4211a1200, 0x19746e0, 0xc4211a1200, 0x181f280, 0xc422c8e800, 0xc420316de0, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc420206170, 0xc4211a1200, 0xc422c8e800, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.call(0xc420316de0, 0xc420206478, 0x13, 0x1bda9a0, 0x4, 0xc420fd9d18, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.Call(0xc420316de0, 0xc420206478, 0x13, 0xc420fd9d18, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*service).call(0xc4202b1100, 0xc4202feeb0, 0xc422b82f48, 0x0, 0xc420143100, 0xc4213f0080, 0x19746e0, 0xc4211a1200, 0x16, 0x181f280, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc4202feeb0, 0x1d71b80, 0xc422c8e7c0, 0x5b7ea86c, 0xc4396702cb) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc4203e8000, 0x1be6698, 0xc, 0x19746e0, 0xc42175a840, 0x181f280, 0xc4215c5c40, 0x1d59ee0, 0xc421249150) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ce3980, 0x1dcd6500, 0x24, 0xc422fa96e0, 0x24, 0x1, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:158 +0x1bb Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ce3980) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 166 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420318600, 0xc4209ff000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc4209d2540) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc4209d2540, 0xc420042500, 0xc420e684d8, 0xc420e684e0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4227978c0, 0xc42188f410) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421d6d7a0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4209d25a0, 0x16e4ba0, 0xc422797b90) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4209d25a0, 0x16e4ba0, 0xc422797b90, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422ce3e80, 0x16e4ba0, 0xc422797b90, 0x1, 0xc422797b90) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc422ce3e80, 0xc422797b90, 0x19746e0, 0xc420f6c240) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc422ce3e80, 0x1be6698, 0xc, 0x19746e0, 0xc420f6c240, 0x181f280, 0xc422c68140, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc420316300, 0xc42003fa51, 0x6, 0x1d62b00, 0xc420ba0360, 0x1, 0x1be6698, 0xc, 0x19746e0, 0xc420f6c240, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc4203e8000, 0xc4204ca1c0, 0x1be6698, 0xc, 0x19746e0, 0xc420f6c240, 0x181f280, 0xc422c68140, 0x1d59ee0, 0x1947b20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc4203e8000, 0x1be6698, 0xc, 0x1d76b80, 0xc420f6c240, 0x19746e0, 0xc420f6c240, 0x181f280, 0xc422c68140, 0xc420316de0, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc420206170, 0xc420f6c240, 0xc422c68140, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.call(0xc420316de0, 0xc420206478, 0x13, 0x1bda9a0, 0x4, 0xc42188fd18, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.Call(0xc420316de0, 0xc420206478, 0x13, 0xc42188fd18, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*service).call(0xc4202b1100, 0xc4202feeb0, 0xc4229fa3f8, 0x0, 0xc420143100, 0xc420cde040, 0x19746e0, 0xc420f6c240, 0x16, 0x181f280, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc4202feeb0, 0x1d71b80, 0xc422c680c0, 0x5b7ea86c, 0xc439a3412c) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc4203e8000, 0x1be6698, 0xc, 0x19746e0, 0xc4211a0540, 0x181f280, 0xc422c8e240, 0x1d59ee0, 0xc420627100) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ce39e0, 0x1dcd6500, 0x24, 0xc4230a1500, 0x24, 0xc420068f00, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:158 +0x1bb Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ce39e0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 167 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a7b0, 0x72, 0xc4213f9490) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc42039cd18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc42039cd18, 0xc421c14000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc42039cd00, 0xc421c14000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc42039cd00, 0xc421c14000, 0x1000, 0x1000, 0x1, 0xc421c14000, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc420210d80, 0xc421c14000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc420ca2480, 0xc4222aca9c, 0x1, 0x1, 0x1000, 0x1000, 0xc4213f9698) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).readTCP(0xc4205143c0, 0x1d7d320, 0xc420210d80, 0x2b96de0, 0x0, 0x0, 0x2b96de0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:912 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc4205143c0, 0xc42246a5b0, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:772 +0x357 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc4205143c0, 0xc42246a5b0, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc4205143c0, 0xc42023b000, 0x14, 0x20, 0x3, 0x3, 0xc4213db780) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc420434d80, 0xc42023b000, 0x14, 0x20, 0xc4213db701, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc4203e8000, 0xc42023b000, 0x14, 0x20, 0xc4213db780, 0x10, 0xc420bdc5e0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc42055a740, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc4213cca20, 0xc4203e8000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 168 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4203e8000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 169 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4203e8000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 170 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3ae30, 0x72, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc42058b418, 0x72, 0xc420083c00, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc42058b418, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Accept(0xc42058b400, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).accept(0xc42058b400, 0x1, 0x1, 0xc420617f20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*TCPListener).accept(0xc420206d98, 0x406b4a, 0xc420fbf9e0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*TCPListener).Accept(0xc420206d98, 0xc420fbf9e0, 0x0, 0x3f800000, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4203e8000, 0x1d71840, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 171 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc420420000, 0x3b9aca00, 0xc4204a6420) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 172 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4202fee60, 0x3b9aca00, 0xc4204a6420) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 173 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420516120, 0x3b9aca00, 0xc4204a6420) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 174 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc42040ed00, 0x3b9aca00, 0xc4204a6420) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 175 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4203e8000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 176 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3abc0, 0x72, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc420a66298, 0x72, 0xc4202b0300, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc420a66298, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Accept(0xc420a66280, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).accept(0xc420a66280, 0x402037, 0xc4218ccd00, 0x1805ec0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*TCPListener).accept(0xc42000eea0, 0x458e40, 0xc420618e90, 0xc420618e98) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*TCPListener).Accept(0xc42000eea0, 0x1c65af8, 0xc4218ccc80, 0x1d71900, 0xc420cd2180) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.(*Server).Serve(0xc420f89a00, 0x1d6fc00, 0xc42000eea0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.Serve(0x1d6fc00, 0xc42000eea0, 0x1d5df00, 0xc420cd2000, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420ca9200, 0x1d6fc00, 0xc42000eea0, 0xc420068c70, 0xc4213ccae0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 177 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420a66300, 0xc420068c80, 0xc420fbf560) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 178 [sleep]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: time.Sleep(0x55cdbc0f1) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc42021f730, 0xc420a66300) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 181 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.(*persistConn).writeLoop(0xc421370ea0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by net/http.(*Transport).dialConn Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 238 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc4203e8000, 0x1bef5ff, 0x11, 0x1d77180, 0xc420fcbda0, 0x1975b20, 0xc420fcbda0, 0x169e1e0, 0xc420ca4b00, 0xc422902570, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Node).UpdateStatus(0xc420fcb740, 0xc420fcbda0, 0xc420ca4b00, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:321 +0xa9 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.call(0xc420448ae0, 0xc420210cf8, 0x13, 0x1bda9a0, 0x4, 0xc420067d48, 0x3, 0x3, 0x1a99fe0, 0x19, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.Call(0xc420448ae0, 0xc420210cf8, 0x13, 0xc420067d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*service).call(0xc422c45500, 0xc4212b2410, 0xc4222ac220, 0x0, 0xc420ca4900, 0xc421d6dfc0, 0x1975b20, 0xc420fcbda0, 0x16, 0x169e1e0, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc4212b2410, 0x1d72340, 0xc422c45540, 0x0, 0xc420318a80) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc420318a80, 0xc4212b2410) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 237 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42238b970) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 184 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a880, 0x72, 0xc421401ce8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc420971418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc420971418, 0xc420bc5000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc420971400, 0xc420bc5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc420971400, 0xc420bc5000, 0x1000, 0x1000, 0xc421401dd8, 0x1, 0xc421401e08) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc4204a4098, 0xc420bc5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc4213e0840, 0xc4213da460, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadAtLeast(0x1d59aa0, 0xc4213e0840, 0xc4213da460, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadFull(0x1d59aa0, 0xc4213e0840, 0xc4213da460, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4201ee0b0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4201ee0b0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 185 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4201ee0b0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 186 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4201ee0b0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 187 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420512000, 0xc420f66000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc4204f37a0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc4204f37a0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4213ddd40, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4213cf520) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4204f3800, 0x16e4b60, 0xc4213cf580) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4204f3800, 0x16e4b60, 0xc4213cf580, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4212f2640, 0x16e4b60, 0xc4213cf580, 0xc4213cf580, 0xc420cd82a8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4212f2640, 0xc4213cf580, 0x2d, 0xc4206c6920) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequestHeader(0xc420cd8280, 0x1d72340, 0xc4212f2640, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequest(0xc420cd8280, 0x1d72340, 0xc4212f2640, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc420cd8280, 0x1d72340, 0xc4212f2640, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc420512000, 0xc420cd8280) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 125 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3aaf0, 0x72, 0xc420edece8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc420fac498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc420fac498, 0xc4201b0000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc420fac480, 0xc4201b0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc420fac480, 0xc4201b0000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc420207588, 0xc4201b0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc420086ae0, 0xc42136fdc0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadAtLeast(0x1d59aa0, 0xc420086ae0, 0xc42136fdc0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadFull(0x1d59aa0, 0xc420086ae0, 0xc42136fdc0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42031e840, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42031e840) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 126 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42031e840) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 127 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42031e840) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 128 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4201f0c00, 0xc4201b1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc420ca31a0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc420ca31a0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420be7230, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4213f1400) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420ca3200, 0x16e4b60, 0xc4213f1460) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420ca3200, 0x16e4b60, 0xc4213f1460, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4213d94c0, 0x16e4b60, 0xc4213f1460, 0xc4213f1460, 0xc420ce0e88) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4213d94c0, 0xc4213f1460, 0x2d, 0xc420c72160) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequestHeader(0xc420ce0e60, 0x1d72340, 0xc4213d94c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420432a00, 0x8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequest(0xc420ce0e60, 0x1d72340, 0xc4213d94c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc420ce0e60, 0x1d72340, 0xc4213d94c0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc4201f0c00, 0xc420ce0e60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 142 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420511290, 0xc4213fed10, 0x30, 0xf9866a) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420511290, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc42000f2a0, 0xc42073b540) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc42000f2a0, 0xc42073b540) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 191 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a6e0, 0x72, 0xc420edcce8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc420c44d98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc420c44d98, 0xc420ada000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc420c44d80, 0xc420ada000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc420c44d80, 0xc420ada000, 0x1000, 0x1000, 0xc4213d25d8, 0x1, 0xc4213d2608) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc42000f2a0, 0xc420ada000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc4209d2000, 0xc420ae3940, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadAtLeast(0x1d59aa0, 0xc4209d2000, 0xc420ae3940, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadFull(0x1d59aa0, 0xc4209d2000, 0xc420ae3940, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420511290, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420511290) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 192 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420511290) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 193 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420511290) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 210 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4203189c0, 0xc420adb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc4209e8360) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc4209e8360, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42085a3c0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420cd5500) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4209e83c0, 0x16e4b60, 0xc420cd5560) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4209e83c0, 0x16e4b60, 0xc420cd5560, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4212d7680, 0x16e4b60, 0xc420cd5560, 0xc420cd5560, 0xc4207a0578) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4212d7680, 0xc420cd5560, 0x2d, 0xc420eeef60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequestHeader(0xc4207a0550, 0x1d72340, 0xc4212d7680, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420432a00, 0x8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequest(0xc4207a0550, 0x1d72340, 0xc4212d7680, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc4207a0550, 0x1d72340, 0xc4212d7680, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc4203189c0, 0xc4207a0550) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 240 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a540, 0x72, 0xc420c6b858) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc42039d298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc42039d298, 0xc4227ce000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc42039d280, 0xc4227ce000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc42039d280, 0xc4227ce000, 0x1000, 0x1000, 0xc420042500, 0xc422c69d18, 0xc422c69d20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc420210998, 0xc4227ce000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc42128afc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc42128afc0, 0x72, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42109a330, 0xc42039d298) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421d6dd20) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42128b020, 0x16e76e0, 0xc42066ccb0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42128b020, 0x16e76e0, 0xc42066ccb0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.decodeResponse(0xc422c44ec0, 0x182ba60, 0xc4225af900, 0xc42175b920, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:608 +0x5e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).genericRPC(0xc420510160, 0xc4204845d0, 0x24, 0xc4202e1020, 0x11, 0x331fb69d2175b900, 0x182ba00, 0xc42175b920, 0x182ba60, 0xc4225af900, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:406 +0xf9 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).AppendEntries(0xc420510160, 0xc4204845d0, 0x24, 0xc4202e1020, 0x11, 0xc42175b920, 0xc4225af900, 0x1, 0x3) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:379 +0x97 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4200c22c0, 0xc4212029c0, 0x18633f, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:186 +0x189 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200c22c0, 0xc4212029c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:132 +0x3f2 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc4224680c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 290 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200c22c0, 0xc421202900, 0x1d6eb80, 0xc422c45a80, 0xc4230167e0, 0xc4230168a0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc422c45b80) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 108 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422374000, 0xc422f7ecc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 234 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42238b970, 0xc420619d10, 0x30, 0xf9866a) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42238b970, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc4202109a8, 0xc4212b2370) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc4202109a8, 0xc4212b2370) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 217 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4201ee160, 0xc420c6fd10, 0x30, 0xf9866a) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4201ee160, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc4204a4010, 0xc420ff8af0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc4204a4010, 0xc420ff8af0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 218 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a470, 0x72, 0xc420c6ace8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc420a10018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc420a10018, 0xc420afb000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc420a10000, 0xc420afb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc420a10000, 0xc420afb000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc4204a4010, 0xc420afb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc420ff1200, 0xc4204a82e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadAtLeast(0x1d59aa0, 0xc420ff1200, 0xc4204a82e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadFull(0x1d59aa0, 0xc420ff1200, 0xc4204a82e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4201ee160, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4201ee160) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 219 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4201ee160) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 220 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4201ee160) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 221 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4204af800, 0xc4213f3700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420e9d740, 0x1d718c0, 0xc4204af800, 0x0, 0xc4204af800) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4203e8000, 0xc4213f3950, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420449f80, 0xc420ca4100, 0xc420e9d500, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.call(0xc420448840, 0xc4204a46b8, 0x13, 0x1bda9a0, 0x4, 0xc4213f3d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.Call(0xc420448840, 0xc4204a46b8, 0x13, 0xc4213f3d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*service).call(0xc420a74700, 0xc420ff8b90, 0xc420b83928, 0x0, 0xc421149900, 0xc420fcfb40, 0x19758e0, 0xc420ca4100, 0x16, 0x169e020, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc420ff8b90, 0x1d72340, 0xc420a751c0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc421202000, 0xc420ff8b90) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 239 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200c22c0, 0xc421202900, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200c22c0, 0xc421202900) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc4224680a0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 224 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42137faa0, 0xc420fd5700, 0x20, 0x20, 0x169d4e0, 0xc4201eafa0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420faffb0, 0x1d718c0, 0xc42137faa0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4203e8000, 0xc420fd5950, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420ca2c60, 0xc42100c080, 0xc420d92a20, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.call(0xc420448840, 0xc420207890, 0x13, 0x1bda9a0, 0x4, 0xc420066d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: reflect.Value.Call(0xc420448840, 0xc420207890, 0x13, 0xc420066d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*service).call(0xc4213d9480, 0xc420ce0e60, 0xc4205e5a48, 0x0, 0xc4201b4d80, 0xc420677ea0, 0x19758e0, 0xc42100c080, 0x16, 0x169e020, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc420ce0e60, 0x1d72340, 0xc420d1bf00, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc421202180, 0xc420ce0e60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 242 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42044cdc0, 0xc42061ad10, 0x30, 0xf9866a) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42044cdc0, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc4204a4708, 0xc42144ae60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc4204a4708, 0xc42144ae60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 243 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a200, 0x72, 0xc420edbce8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc420dbc518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc420dbc518, 0xc420594000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc420dbc500, 0xc420594000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc420dbc500, 0xc420594000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc4204a4708, 0xc420594000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc421618480, 0xc421922000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadAtLeast(0x1d59aa0, 0xc421618480, 0xc421922000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadFull(0x1d59aa0, 0xc421618480, 0xc421922000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42044cdc0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42044cdc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 244 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42044cdc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 245 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42044cdc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 246 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420512180, 0xc421574000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc420fcaa80) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc420fcaa80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421880720, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4219bf720) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420fcaae0, 0x16e4b60, 0xc4219bf780) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420fcaae0, 0x16e4b60, 0xc4219bf780, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421959340, 0x16e4b60, 0xc4219bf780, 0xc4219bf780, 0xc42144af28) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421959340, 0xc4219bf780, 0x2d, 0xc420a60d00) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequestHeader(0xc42144af00, 0x1d72340, 0xc421959340, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420432a00, 0x8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequest(0xc42144af00, 0x1d72340, 0xc421959340, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc42144af00, 0x1d72340, 0xc421959340, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc420512180, 0xc42144af00) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 291 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc422774960) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 235 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a060, 0x72, 0xc420063ce8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc42021bf18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc42021bf18, 0xc422318000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc42021bf00, 0xc422318000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc42021bf00, 0xc422318000, 0x1000, 0x1000, 0xc420063dd8, 0x1, 0xc420063e08) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc4202109a8, 0xc422318000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc42128b140, 0xc42248c260, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadAtLeast(0x1d59aa0, 0xc42128b140, 0xc42248c260, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadFull(0x1d59aa0, 0xc42128b140, 0xc42248c260, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42238b970, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42238b970) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 199 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42238a2c0, 0xc420ee0d10, 0x30, 0xf9866a) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42238a2c0, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc42000f130, 0xc42299ab90) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc42000f130, 0xc42299ab90) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 200 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a130, 0x72, 0xc423237ce8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc420bfc318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc420bfc318, 0xc423042000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc420bfc300, 0xc423042000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc420bfc300, 0xc423042000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc42000f130, 0xc423042000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc422215b60, 0xc421ad36a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadAtLeast(0x1d59aa0, 0xc422215b60, 0xc421ad36a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadFull(0x1d59aa0, 0xc422215b60, 0xc421ad36a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42238a2c0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42238a2c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 201 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42238a2c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 202 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42238a2c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 203 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4212020c0, 0xc423043000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc422648cc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc422648cc0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4230b8d50, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4227a2f60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422648d20, 0x16e4b60, 0xc4227a2fc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422648d20, 0x16e4b60, 0xc4227a2fc0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421be9600, 0x16e4b60, 0xc4227a2fc0, 0xc4227a2fc0, 0xc42299ac58) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421be9600, 0xc4227a2fc0, 0x2d, 0xc4226ac200) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequestHeader(0xc42299ac30, 0x1d72340, 0xc421be9600, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequest(0xc42299ac30, 0x1d72340, 0xc421be9600, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc42299ac30, 0x1d72340, 0xc421be9600, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc4212020c0, 0xc42299ac30) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 106 [runnable]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc420420000, 0xc4207cab00, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:210 +0x359 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc420420000, 0xc4207cab00) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4203e8000, 0xc420028c01, 0xc420028c80) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4203e8000, 0xc422f7e2a0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4203e8000, 0xc422f7e2a0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc42136e870, 0xc4203e8000, 0xc422f7e2a0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 277 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4201f1500, 0xc4221ad000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).fill(0xc420f5ac60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).ReadByte(0xc420f5ac60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42212baa0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4224bc860) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420f5acc0, 0x16e4b60, 0xc4224bc8c0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420f5acc0, 0x16e4b60, 0xc4224bc8c0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422c23940, 0x16e4b60, 0xc4224bc8c0, 0xc4224bc8c0, 0xc422b7cf28) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422c23940, 0xc4224bc8c0, 0x2d, 0xc42249b740) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequestHeader(0xc422b7cf00, 0x1d72340, 0xc422c23940, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420432a00, 0x8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).readRequest(0xc422b7cf00, 0x1d72340, 0xc422c23940, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/rpc.(*Server).ServeRequest(0xc422b7cf00, 0x1d72340, 0xc422c23940, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d0e0, 0xc4201f1500, 0xc422b7cf00) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 276 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200c22c0, 0xc421202900, 0xc42303afc0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc4224bc7a0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 107 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422374000, 0xc422f7e720) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 257 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4222ec160) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 258 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4222ec160) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 260 [runnable]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: strings.Count(0x162f668, 0x10, 0x1bd974b, 0x1, 0x40) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/strings/strings_amd64.go:83 +0xaf Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: strings.genSplit(0x162f668, 0x10, 0x1bd974b, 0x1, 0x0, 0xffffffffffffffff, 0x1a5dc20, 0xc420bb9401, 0xc422bb4280) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/strings/strings.go:243 +0x23e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: strings.Split(0x162f668, 0x10, 0x1bd974b, 0x1, 0x1, 0xc420bb94d8, 0x8f43a4) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/strings/strings.go:298 +0x5b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec.parseStructFieldInfo(0x162f65a, 0x6, 0x162f668, 0x10, 0x10) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec/helper.go:668 +0xdc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec.(*TypeInfos).rget(0xc420083600, 0x1d87cc0, 0x196f2c0, 0x196f2c0, 0x0, 0x0, 0x0, 0x0, 0xc420bb9738) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec/helper.go:976 +0x5bb Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec.(*TypeInfos).get(0xc420083600, 0x196f2c0, 0x1d87cc0, 0x196f2c0, 0xc422adf800) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec/helper.go:872 +0x82b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec.(*BasicHandle).getTypeInfo(0x2aba440, 0x196f2c0, 0x1d87cc0, 0x196f2c0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec/helper.go:349 +0x97 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec.(*Encoder).getEncFn(0xc420506780, 0x196f2c0, 0x1d87cc0, 0x196f2c0, 0x100, 0x199) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec/encode.go:1274 +0x15e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec.(*Encoder).doEncodeValue(0xc420506780, 0x196f2c0, 0xc42066caf0, 0x199, 0x0, 0x0, 0x100) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec/encode.go:1221 +0x159 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec.(*Encoder).encodeI(0xc420506780, 0x181eb00, 0xc42066caf0, 0x100) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec/encode.go:1231 +0xc9 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec.(*Encoder).encode(0xc420506780, 0x181eb00, 0xc42066caf0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec/encode.go:1167 +0x13e Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec.(*Encoder).Encode(0xc420506780, 0x181eb00, 0xc42066caf0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/ugorji/go/codec/encode.go:1072 +0x71 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc420f6de60, 0xc4209b2800) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:327 +0x435 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.HandlerFunc.ServeHTTP(0xc4213cebe0, 0x1d6df80, 0xc420f6de60, 0xc4209b2800) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.(*ServeMux).ServeHTTP(0xc4213ccae0, 0x1d6df80, 0xc420f6de60, 0xc4209b2800) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc422f24460, 0xc4209b2800) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.HandlerFunc.ServeHTTP(0xc420cd2000, 0x1d6fec0, 0xc422f24460, 0xc4209b2800) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.serverHandler.ServeHTTP(0xc420f89a00, 0x1d6fec0, 0xc422f24460, 0xc4209b2800) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.(*conn).serve(0xc4218ccc80, 0x1d71840, 0xc422c69680) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by net/http.(*Server).Serve Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 261 [IO wait]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f2862d3a610, 0x72, 0xc42055be58) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc4204c9b18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc4204c9b18, 0xc42255bc00, 0x1, 0x1) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc4204c9b00, 0xc42255bc01, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc4204c9b00, 0xc42255bc01, 0x1, 0x1, 0xc42055bfa0, 0x0, 0xc420ce3800) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc422144308, 0xc42255bc01, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net/http.(*connReader).backgroundRead(0xc42255bbf0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 236 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42238b970) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 263 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200c22c0, 0xc4212029c0, 0xc423026600) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc4224f0e60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 264 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4214c64d0, 0xc420edad10, 0x30, 0xf9866a) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4214c64d0, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420ca9140) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc422144318, 0xc422b7ce60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420ca9140, 0x1d7d320, 0xc422144318, 0xc422b7ce60) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 265 [runnable]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.runtime_pollWait(0x7f28600247c0, 0x72, 0xc423233ce8) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).wait(0xc4225d4018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*pollDesc).waitRead(0xc4225d4018, 0xc421806000, 0x1000, 0x1000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: internal/poll.(*FD).Read(0xc4225d4000, 0xc421806000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*netFD).Read(0xc4225d4000, 0xc421806000, 0x1000, 0x1000, 0x96, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: net.(*conn).Read(0xc422144318, 0xc421806000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: bufio.(*Reader).Read(0xc42175baa0, 0xc4223d8050, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadAtLeast(0x1d59aa0, 0xc42175baa0, 0xc4223d8050, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: io.ReadFull(0x1d59aa0, 0xc42175baa0, 0xc4223d8050, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4214c64d0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4214c64d0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 266 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4214c64d0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 267 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4214c64d0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 269 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420458b40) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 270 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420458b40) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 271 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4202fee60, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4203e8000) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 273 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420516120) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 306 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420516120) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 307 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc42244fcb0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 308 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422bf5440, 0xc422fbece8, 0x20, 0x20, 0x18, 0xc422468b40) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422311ef0, 0x1d71840, 0xc422bf5440, 0xc422468b40, 0x184987) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420028c80, 0xc421889ed0, 0x184987, 0x1d71840, 0xc422bf5440, 0xc42021a460, 0xc42244fc80, 0xc420c97050, 0x24, 0x2bba400) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc42021a400, 0x1d71840, 0xc422bf5440, 0x184987, 0xc422c45cc0, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc42021a400, 0x1d71840, 0xc422bf5440) Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: goroutine 309 [select]: Aug 23 15:28:29 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422bf5580, 0xc420cc1770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422494510, 0x1d71840, 0xc422bf5580, 0x0, 0x1849b5) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420028c80, 0xc420cc1960, 0x1849b5, 0x1d71840, 0xc422bf5580, 0x552bde, 0xc420cc1950, 0xc420cc1970, 0xf33ba5, 0xc42044e988) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc42044e930, 0x1d71840, 0xc422bf5580, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc42044e930) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: goroutine 310 [select]: Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422bf5540, 0xc423097a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224942d0, 0x1d71840, 0xc422bf5540, 0xc4224943c0, 0x1833a4) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420028c80, 0xc423097c68, 0x1833a4, 0x1d71840, 0xc422bf5540, 0x552bde, 0xc4202febe0, 0x2, 0xc420ef3000, 0x40) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422bf55c0, 0x1833a4, 0x3b, 0xc4230d8de8, 0x1, 0x1) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422bf55c0) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: goroutine 311 [select]: Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422bf5600) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: goroutine 312 [select]: Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4205145a0, 0x1d71840, 0xc422bf5540) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: goroutine 293 [select]: Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422c45d80, 0xc4230d8d28, 0x20, 0x20, 0xc42017f000, 0x1bd9a98) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224944e0, 0x1d71840, 0xc422c45d80, 0xc422468ec0, 0x1849b5) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420028c80, 0xc423099f20, 0x1849b5, 0x1d71840, 0xc422c45d80, 0x1d55e50, 0x0, 0x0, 0x0, 0xc420fbf9e0) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc422774960, 0x1849b5, 0x63c8dd, 0x1, 0x0, 0x0, 0x0, 0xc42147c120) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc422774960, 0x1849b5, 0xc420448de0) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25822]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:28:30 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:28:30 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:28:30 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: ==> Starting Nomad agent... Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: ==> Nomad agent configuration: Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: Client: false Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: Log Level: INFO Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: Region: amrv01 (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: Server: true Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: Version: 0.8.4 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.469655 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: Attempting re-join to previously known node: consulnomad-03.aaf01: 172.26.1.78:4648 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.469769 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.512634 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650297 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [WARN] memberlist: Refuting an alive message Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650323 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650567 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650584 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650826 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650842 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650855 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650867 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650878 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650889 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650900 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.650912 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.651195 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.651210 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30 [INFO] serf: Re-joined to previously known node: consulnomad-03.aaf01: 172.26.1.78:4648 Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.651221 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.651242 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.651254 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.651265 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:28:30 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:30.651276 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:28:31 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:31 [WARN] raft: Rejecting vote request from 172.24.10.89:4647 since our last term is greater (46, 45) Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 48 Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [INFO] raft: Election won. Tally: 2 Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32.513677 [INFO] nomad: cluster leadership acquired Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [INFO] raft: pipelining replication to peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [WARN] raft: AppendEntries to {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} rejected, sending older logs (next: 1598272) Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:32 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:33 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: 2018/08/23 15:28:33 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: fatal error: concurrent map writes Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 217 [running]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc420c63978 sp=0xc420c63958 pc=0x42d161 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: runtime.mapassign(0x1844be0, 0xc420554450, 0xc420c63aa8, 0x2bba400) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/hashmap.go:519 +0x56e fp=0xc420c63a08 sp=0xc420c63978 pc=0x40a41e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc420556240, 0x1d6dd40, 0xc42000ea28, 0x3776fd6, 0xed3108c9b, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:99 +0x1d1 fp=0xc420c63af8 sp=0xc420c63a08 pc=0xf20d91 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4204a6780, 0xc420a4a160, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc420c63b60 sp=0xc420c63af8 pc=0xf5388e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4204a6780, 0xc420a4a160) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc420c63b98 sp=0xc420c63b60 pc=0xf534b3 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4200e0280, 0xc4203d4501, 0xc4203d4500) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc420c63c88 sp=0xc420c63b98 pc=0xf77602 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4200e0280, 0xc420edf020, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc420c63d38 sp=0xc420c63c88 pc=0xf76fff Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4200e0280, 0xc420edf020) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc420c63fa0 sp=0xc420c63d38 pc=0xf76b90 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc42075a160, 0xc4200e0280, 0xc420edf020) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc420c63fc8 sp=0xc420c63fa0 pc=0xfb8e9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: runtime.goexit() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420c63fd0 sp=0xc420c63fc8 pc=0x45c711 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 1 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420318af0, 0xc420492280) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420318af0, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc420477e00, 0xc420477e00, 0xc4205016e0, 0x37) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204e7f68) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: main.main() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 5 [syscall]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: os/signal.signal_recv(0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: os/signal.loop() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by os/signal.init.0 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 53 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc420086420) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 52 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420550300) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 51 [sleep]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: time.Sleep(0x3b9aca00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc4203e8000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 69 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc42054e080, 0x1d71840, 0xc4201fd700) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 170 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420550900, 0xc420c02000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).fill(0xc4205174a0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).ReadByte(0xc4205174a0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d3c300, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42109bc20) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420517500, 0x16e4b60, 0xc42109bc80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420517500, 0x16e4b60, 0xc42109bc80, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420c73980, 0x16e4b60, 0xc42109bc80, 0xc42109bc80, 0xc4203ca528) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420c73980, 0xc42109bc80, 0x2d, 0xc420704f80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequestHeader(0xc4203ca500, 0x1d72340, 0xc420c73980, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42039b180, 0x8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequest(0xc4203ca500, 0x1d72340, 0xc420c73980, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).ServeRequest(0xc4203ca500, 0x1d72340, 0xc420c73980, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d0e0, 0xc420550900, 0xc4203ca500) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 211 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4204408f0, 0xc420f3ad10, 0x30, 0xf9866a) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4204408f0, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420e1a000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d320, 0xc4202028d8, 0xc4203cbc20) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d320, 0xc4202028d8, 0xc4203cbc20) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 50 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc420082180) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 12 [select, locked to thread]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: runtime.selectgo(0xc420055f50, 0xc4202e8900) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: runtime.ensureSigM.func1() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: runtime.goexit() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 55 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4205624e0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4205548a0, 0xc42054c150) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 119 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420d25710) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 15 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420504000, 0x1c636a0, 0xc4203e8160, 0x1d71840, 0xc4204fd100) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4203e8160) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 13 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db0f00, 0x72, 0xc4205a39a8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc420502698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitRead(0xc420502698, 0xc4205ac000, 0x1000, 0x1000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).Read(0xc420502680, 0xc4205ac000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).Read(0xc420502680, 0xc4205ac000, 0x1000, 0x1000, 0x458600, 0xc420fb3980, 0x4) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*conn).Read(0xc420518600, 0xc4205ac000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/http.(*persistConn).Read(0xc4200ccb40, 0xc4205ac000, 0x1000, 0x1000, 0xc4205a3b98, 0x405675, 0xc4202e8a80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).fill(0xc420383320) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).Peek(0xc420383320, 0x1, 0x0, 0x0, 0x1, 0xc420d7c840, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/http.(*persistConn).readLoop(0xc4200ccb40) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by net/http.(*Transport).dialConn Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 145 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc42079ac80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 117 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4205b22c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205b22c0, 0xc4206a05e0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 179 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420440790) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 65 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420440790, 0xc4205a5d10, 0x30, 0xf9866a) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420440790, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420e1a000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d320, 0xc420202068, 0xc4203ca230) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d320, 0xc420202068, 0xc4203ca230) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 227 [semacquire]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: sync.runtime_SemacquireMutex(0xc4204a680c, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: sync.(*Mutex).Lock(0xc4204a6808) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: sync.(*RWMutex).Lock(0xc4204a6808) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4204a6780, 0x1d71840, 0xc422ed9d00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:28:33 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 116 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4205b22c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4205b22c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4205b22c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205b22c0, 0xc4206a05c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 14 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/http.(*persistConn).writeLoop(0xc4200ccb40) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by net/http.(*Transport).dialConn Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 118 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4205b22c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205b22c0, 0xc4206a05f0) Aug 23 15:28:33 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 144 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc42079ac80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 186 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420440840, 0xc420f1dd10, 0x30, 0xf9866a) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420440840, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420e1a000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d320, 0xc420202568, 0xc4203cb720) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d320, 0xc420202568, 0xc4203cb720) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 203 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4205b22c0, 0xc420bcc300, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4205b22c0, 0xc420bcc300) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205b22c0, 0xc420720ec0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 178 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db0bc0, 0x72, 0xc420060ce8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc420a15318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitRead(0xc420a15318, 0xc421042000, 0x1000, 0x1000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).Read(0xc420a15300, 0xc421042000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).Read(0xc420a15300, 0xc421042000, 0x1000, 0x1000, 0xc420060dd8, 0x0, 0xc420060e08) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*conn).Read(0xc420202068, 0xc421042000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).Read(0xc4205e82a0, 0xc42075a2f0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: io.ReadAtLeast(0x1d59aa0, 0xc4205e82a0, 0xc42075a2f0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: io.ReadFull(0x1d59aa0, 0xc4205e82a0, 0xc42075a2f0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420440790, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420440790) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 226 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4200a8a50, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4200e0280) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 120 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc42039c0d0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 121 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db0d60, 0x72, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc420d96298, 0x72, 0xc4201fd700, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitRead(0xc420d96298, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).Accept(0xc420d96280, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).accept(0xc420d96280, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*TCPListener).accept(0xc420518260, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*TCPListener).AcceptTCP(0xc420518260, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4205e63c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 122 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db0c90, 0x72, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc420d96318, 0x72, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitRead(0xc420d96318, 0xc422c64000, 0x10000, 0x10000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).ReadFrom(0xc420d96300, 0xc422c64000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).readFrom(0xc420d96300, 0xc422c64000, 0x10000, 0x10000, 0xc4202ebe01, 0x0, 0xc420f1ee60, 0x41274d, 0xc422c64000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*UDPConn).readFrom(0xc420518278, 0xc422c64000, 0x10000, 0x10000, 0x101, 0xc420f1ee18, 0xc420f1ee90, 0x443cf7) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*UDPConn).ReadFrom(0xc420518278, 0xc422c64000, 0x10000, 0x10000, 0x10000, 0x10000, 0xbc510, 0xc512ac5f, 0x2b96de0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4205e63c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 123 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4205e63c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 124 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4205e63c0, 0x12a05f200, 0xc420b61ce0, 0xc420d63740, 0xc4206a0c70) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 125 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4205e63c0, 0xc420d63740) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 126 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4205e63c0, 0x1dcd6500, 0xc420b61d40, 0xc420d63740, 0xc4206a0c80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 127 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc420001e00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 128 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc420001e00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 129 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420001e00, 0x1bdd09a, 0x6, 0xc420d25c80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 146 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420001e00, 0x1bdba6c, 0x5, 0xc420d25d10) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 147 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420001e00, 0x1bdbc6a, 0x5, 0xc420d25d70) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 132 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4205503c0, 0xc42094c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).fill(0xc420dfc300) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).ReadByte(0xc420dfc300, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420e2b7a0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420734b60) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420dfc360, 0x16e4b60, 0xc420734bc0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420dfc360, 0x16e4b60, 0xc420734bc0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420eeac00, 0x16e4b60, 0xc420734bc0, 0xc420734bc0, 0xc4203cb7e8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420eeac00, 0xc420734bc0, 0x2d, 0xc422cbbd00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequestHeader(0xc4203cb7c0, 0x1d72340, 0xc420eeac00, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42039b180, 0x8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequest(0xc4203cb7c0, 0x1d72340, 0xc420eeac00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).ServeRequest(0xc4203cb7c0, 0x1d72340, 0xc420eeac00, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d0e0, 0xc4205503c0, 0xc4203cb7c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 149 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420b61da0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420b61da0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420b61da0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 150 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420b61e00, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420b61e00, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420b61e00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 151 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420b61e60, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420b61e60, 0x1dcd6500, 0x0, 0x0, 0x185b9a, 0xb, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420b61e60) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 152 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420b61ec0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420b61ec0, 0x1dcd6500, 0x0, 0xc420f1b900, 0x279, 0x279, 0x7fe67f05d000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420b61ec0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 153 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db07b0, 0x77, 0xc420fb3980) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc420eec418, 0x77, 0xc420f2b000, 0xc420f2b028, 0x436544) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitWrite(0xc420eec418, 0xc420f2b000, 0x28, 0x611b26) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).WaitWrite(0xc420eec400, 0x1c65988, 0x1d718c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:440 +0x37 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).connect(0xc420eec400, 0x1d718c0, 0xc422c850e0, 0x0, 0x0, 0x1d5c800, 0xc4223854e0, 0x0, 0x0, 0x0, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:152 +0x299 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).dial(0xc420eec400, 0x1d718c0, 0xc422c850e0, 0x1d7a200, 0x0, 0x1d7a200, 0xc422c20780, 0xc420f2b248, 0x62f2be) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/sock_posix.go:142 +0xe9 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.socket(0x1d718c0, 0xc422c850e0, 0x1bda392, 0x3, 0x2, 0x1, 0x0, 0x0, 0x1d7a200, 0x0, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/sock_posix.go:93 +0x1a6 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.internetSocket(0x1d718c0, 0xc422c850e0, 0x1bda392, 0x3, 0x1d7a200, 0x0, 0x1d7a200, 0xc422c20780, 0x1, 0x0, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/ipsock_posix.go:141 +0x129 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.doDialTCP(0x1d718c0, 0xc422c850e0, 0x1bda392, 0x3, 0x0, 0xc422c20780, 0x2bb8aa0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/tcpsock_posix.go:62 +0xb9 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.dialTCP(0x1d718c0, 0xc422c850e0, 0x1bda392, 0x3, 0x0, 0xc422c20780, 0x7c559601d, 0x2b96de0, 0x6fc238685) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/tcpsock_posix.go:58 +0xe4 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.dialSingle(0x1d718c0, 0xc422c850e0, 0xc420eec300, 0x1d62b00, 0xc422c20780, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/dial.go:547 +0x375 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.dialSerial(0x1d718c0, 0xc422c850e0, 0xc420eec300, 0xc4204cf850, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/dial.go:515 +0x22d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*Dialer).DialContext(0xc420f2b848, 0x1d71880, 0xc42003e028, 0x1bda392, 0x3, 0xc422702cd0, 0x10, 0x0, 0x0, 0x0, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/dial.go:397 +0x678 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*Dialer).Dial(0xc420f2b848, 0x1bda392, 0x3, 0xc422702cd0, 0x10, 0x20, 0x20, 0x17503e0, 0x2ade3c8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/dial.go:320 +0x75 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc4205e63c0, 0xc422702c70, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:758 +0x14e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc4205e63c0, 0xc422702c70, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc4205e63c0, 0xc420da8000, 0x14, 0x20, 0x3, 0x3, 0xc4206c4300) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc420001e00, 0xc420da8000, 0x14, 0x20, 0xc4206c4301, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc4200e0280, 0xc420da8000, 0x14, 0x20, 0xc4206c4300, 0x10, 0xc4207200e0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc420f19f40, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420ed6a80, 0xc4200e0280) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 154 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4200e0280) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 155 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4200e0280) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 156 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db0e30, 0x72, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc4205d2e98, 0x72, 0xc4201fc000, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitRead(0xc4205d2e98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).Accept(0xc4205d2e80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).accept(0xc4205d2e80, 0x1, 0x1, 0xc4205cef20) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*TCPListener).accept(0xc42000f0c0, 0x406b4a, 0xc420ef9080, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*TCPListener).Accept(0xc42000f0c0, 0xc420ef9080, 0x0, 0x3f800000, 0xc420e1a000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4200e0280, 0x1d71840, 0xc420e1a000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 157 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4204a6780, 0x3b9aca00, 0xc42054a3c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 158 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4200a8a50, 0x3b9aca00, 0xc42054a3c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 159 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc42055c120, 0x3b9aca00, 0xc42054a3c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 160 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4205624e0, 0x3b9aca00, 0xc42054a3c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 161 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4200e0280) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 162 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db0af0, 0x72, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc420d97118, 0x72, 0xc420952900, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitRead(0xc420d97118, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).Accept(0xc420d97100, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).accept(0xc420d97100, 0x412d88, 0x30, 0x1a1fac0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*TCPListener).accept(0xc420518e68, 0x5fea4a, 0x1a1fac0, 0xc420d3c240) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*TCPListener).Accept(0xc420518e68, 0xc42003e028, 0x1830320, 0x2ab0680, 0x1b7b4c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/http.(*Server).Serve(0xc420f5e0d0, 0x1d6fc00, 0xc420518e68, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/http.Serve(0x1d6fc00, 0xc420518e68, 0x1d5df00, 0xc420d3c180, 0xc420ce4020, 0xc420fb0690) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420e1a0c0, 0x1d6fc00, 0xc420518e68, 0xc4206a11c0, 0xc420ed6b40) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 163 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420d97400, 0xc4206a11d0, 0xc420d63860) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 164 [sleep]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: time.Sleep(0x604cffe91) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc420318af0, 0xc420d97400) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 180 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420440790) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 208 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).connect.func2(0x1d718c0, 0xc422c850e0, 0xc420eec400, 0xc422be20c0, 0xc422be2060) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:129 +0xde Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by net.(*netFD).connect Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:128 +0x286 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 225 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc421b714c0, 0xc420f3dd28, 0x20, 0x20, 0xc4201e6400, 0x1bd9a98) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421f9dfb0, 0x1d71840, 0xc421b714c0, 0xc4216d4ce0, 0x1849b5) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203d4500, 0xc421105f20, 0x1849b5, 0x1d71840, 0xc421b714c0, 0x1d55e50, 0x0, 0x0, 0x4, 0xc420fafb80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc423004dc0, 0x1849b5, 0x63c8dd, 0x1, 0x0, 0x0, 0x0, 0xc42126ac60) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc423004dc0, 0x1849b5, 0xc4227565a0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 222 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc421f9d440, 0xc422cb8180) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 140 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4205b22c0, 0xc420bcc300, 0xc420e2ea20) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205b22c0, 0xc4207642a0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 218 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420d68800) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 47 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db0950, 0x72, 0xc4205c89a8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc420974218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitRead(0xc420974218, 0xc420bef000, 0x1000, 0x1000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).Read(0xc420974200, 0xc420bef000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).Read(0xc420974200, 0xc420bef000, 0x1000, 0x1000, 0x458600, 0xc4202ea180, 0x4) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*conn).Read(0xc42020c1f0, 0xc420bef000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/http.(*persistConn).Read(0xc420c1e000, 0xc420bef000, 0x1000, 0x1000, 0xc4205c8b98, 0x405675, 0xc420ecad80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).fill(0xc420961ec0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).Peek(0xc420961ec0, 0x1, 0x0, 0x0, 0x1, 0xc420eca240, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/http.(*persistConn).readLoop(0xc420c1e000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by net/http.(*Transport).dialConn Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 48 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/http.(*persistConn).writeLoop(0xc420c1e000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by net/http.(*Transport).dialConn Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 187 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db0a20, 0x72, 0xc420f40ce8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc4212cc818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitRead(0xc4212cc818, 0xc420973000, 0x1000, 0x1000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).Read(0xc4212cc800, 0xc420973000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).Read(0xc4212cc800, 0xc420973000, 0x1000, 0x1000, 0x8e, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*conn).Read(0xc420202568, 0xc420973000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).Read(0xc420faf7a0, 0xc42079c600, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: io.ReadAtLeast(0x1d59aa0, 0xc420faf7a0, 0xc42079c600, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: io.ReadFull(0x1d59aa0, 0xc420faf7a0, 0xc42079c600, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420440840, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420440840) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 188 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420440840) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 189 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420440840) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 221 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc421f9d440, 0xc422cb8120) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 223 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc423004dc0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 193 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420bcc180, 0xc42094e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).fill(0xc4209a83c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).ReadByte(0xc4209a83c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420198f60, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42076aac0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4209a8420, 0x16e4b60, 0xc42076ab20) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4209a8420, 0x16e4b60, 0xc42076ab20, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420b34580, 0x16e4b60, 0xc42076ab20, 0xc42076ab20, 0xc4203ca528) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420b34580, 0xc42076ab20, 0x2d, 0xc421e5b140) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequestHeader(0xc4203ca500, 0x1d72340, 0xc420b34580, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42039b180, 0x8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequest(0xc4203ca500, 0x1d72340, 0xc420b34580, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).ServeRequest(0xc4203ca500, 0x1d72340, 0xc420b34580, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d0e0, 0xc420bcc180, 0xc4203ca500) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 135 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420232d80, 0xc42124c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).fill(0xc420b6bf80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).ReadByte(0xc420b6bf80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420b69d10, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42076be40) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42124e000, 0x16e4b60, 0xc42076bea0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42124e000, 0x16e4b60, 0xc42076bea0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420b353c0, 0x16e4b60, 0xc42076bea0, 0xc42076bea0, 0xc4203ca528) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420b353c0, 0xc42076bea0, 0x2d, 0xc4213cac00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequestHeader(0xc4203ca500, 0x1d72340, 0xc420b353c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42039b180, 0x8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequest(0xc4203ca500, 0x1d72340, 0xc420b353c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).ServeRequest(0xc4203ca500, 0x1d72340, 0xc420b353c0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d0e0, 0xc420232d80, 0xc4203ca500) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 212 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db0880, 0x72, 0xc420c4ece8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc420197818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitRead(0xc420197818, 0xc42093a000, 0x1000, 0x1000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).Read(0xc420197800, 0xc42093a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).Read(0xc420197800, 0xc42093a000, 0x1000, 0x1000, 0xc4204bfdd8, 0x1, 0xc4204bfe08) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*conn).Read(0xc4202028d8, 0xc42093a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).Read(0xc4209a8840, 0xc4206c4080, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: io.ReadAtLeast(0x1d59aa0, 0xc4209a8840, 0xc4206c4080, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: io.ReadFull(0x1d59aa0, 0xc4209a8840, 0xc4206c4080, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4204408f0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4204408f0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 213 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4204408f0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 214 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4204408f0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 215 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420550480, 0xc42093b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).fill(0xc420b6ac60) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).ReadByte(0xc420b6ac60, 0x101000000000016, 0x0, 0x2d) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420b689c0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42076af40) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420b6acc0, 0x16e4b60, 0xc42076afa0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420b6acc0, 0x16e4b60, 0xc42076afa0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420b34b40, 0x16e4b60, 0xc42076afa0, 0xc42076afa0, 0xc4203cbce8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420b34b40, 0xc42076afa0, 0x2d, 0xc4223126e0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequestHeader(0xc4203cbcc0, 0x1d72340, 0xc420b34b40, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42039b180, 0x8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequest(0xc4203cbcc0, 0x1d72340, 0xc420b34b40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).ServeRequest(0xc4203cbcc0, 0x1d72340, 0xc420b34b40, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d0e0, 0xc420550480, 0xc4203cbcc0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 202 [IO wait]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.runtime_pollWait(0x7fe6a4db06e0, 0x72, 0xc42104bac8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).wait(0xc420e76018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*pollDesc).waitRead(0xc420e76018, 0xc4205f7000, 0x1000, 0x1000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: internal/poll.(*FD).Read(0xc420e76000, 0xc4205f7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*netFD).Read(0xc420e76000, 0xc4205f7000, 0x1000, 0x1000, 0xc4203f6960, 0xc420452380, 0xc4204520e0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net.(*conn).Read(0xc420518028, 0xc4205f7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).fill(0xc420226e40) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).ReadByte(0xc420226e40, 0x3, 0xc420044a70, 0xc420044a00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleCommand(0xc4203e8160, 0xc420226e40, 0xc420226ea0, 0xc4204520e0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:515 +0x43 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleConn(0xc4203e8160, 0x1d71840, 0xc4204fd100, 0x1d7d320, 0xc420518028) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:499 +0x23d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:477 +0x1ce Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 168 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4205b22c0, 0xc420bcc3c0, 0xc420eaa000) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205b22c0, 0xc42109b6a0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 204 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4205b22c0, 0xc420bcc3c0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4205b22c0, 0xc420bcc3c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205b22c0, 0xc420720ee0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 143 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420bcce40, 0xc421043000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).fill(0xc42115c6c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: bufio.(*Reader).ReadByte(0xc42115c6c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4208a1470, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206566c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42115c720, 0x16e4b60, 0xc420656760) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42115c720, 0x16e4b60, 0xc420656760, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420af6d80, 0x16e4b60, 0xc420656760, 0xc420656760, 0xc4203cb7e8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420af6d80, 0xc420656760, 0x2d, 0xc421b41500) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequestHeader(0xc4203cb7c0, 0x1d72340, 0xc420af6d80, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42039b180, 0x8) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).readRequest(0xc4203cb7c0, 0x1d72340, 0xc420af6d80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: net/rpc.(*Server).ServeRequest(0xc4203cb7c0, 0x1d72340, 0xc420af6d80, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420e1a000, 0x1d7d0e0, 0xc420bcce40, 0xc4203cb7c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 219 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4205b22c0, 0xc420bcc300, 0x1d6eb80, 0xc420d68800, 0xc420edf560, 0xc420edf8c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205b22c0, 0xc420d68840) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 205 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc42134bb80) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 206 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4205b22c0, 0xc420bcc3c0, 0x1d6eb80, 0xc42134bb80, 0xc4208e4f00, 0xc4208e5020) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205b22c0, 0xc42134bbc0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 228 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc42055c120) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 229 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc42055c120) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 230 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc422c9c9c0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 231 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422ed9dc0, 0xc420f1bce8, 0x20, 0x20, 0x18, 0xc4216d48a0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421f9d740, 0x1d71840, 0xc422ed9dc0, 0xc4216d48a0, 0x184987) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203d4500, 0xc4212d1ed0, 0x184987, 0x1d71840, 0xc422ed9dc0, 0xc420d96860, 0xc422c9c990, 0xc4207a1620, 0x24, 0x2bba400) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420d96800, 0x1d71840, 0xc422ed9dc0, 0x184987, 0xc421b70fc0, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420d96800, 0x1d71840, 0xc422ed9dc0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 232 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422ed9e40, 0xc42104d770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421e6d440, 0x1d71840, 0xc422ed9e40, 0x0, 0x1849b5) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203d4500, 0xc42104d960, 0x1849b5, 0x1d71840, 0xc422ed9e40, 0x552bde, 0xc42104d950, 0xc42104d970, 0xf33ba5, 0xc4226b5788) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc4226b5730, 0x1d71840, 0xc422ed9e40, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc4226b5730) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 233 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422ed9e00, 0xc421101a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421a1da70, 0x1d71840, 0xc422ed9e00, 0xc421a1dd40, 0x1833a4) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203d4500, 0xc421101c68, 0x1833a4, 0x1d71840, 0xc422ed9e00, 0x552bde, 0xc4200a81e0, 0x2, 0xc420dc4780, 0x40) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422ed9ec0, 0x1833a4, 0x3b, 0xc4205a7de8, 0x1, 0x1) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422ed9ec0) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 234 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422ed9f00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: goroutine 235 [select]: Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4205e65a0, 0x1d71840, 0xc422ed9e00) Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:28:33 ip-172-24-20-236 nomad.sh[25843]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:28:33 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:28:33 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:28:33 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: ==> Starting Nomad agent... Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: ==> Nomad agent configuration: Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: Client: false Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: Log Level: INFO Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: Region: amrv01 (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: Server: true Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: Version: 0.8.4 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.222933 [INFO] nomad: starting 4 scheduling worker(s) for [system service batch _core] Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: Attempting re-join to previously known node: consulnomad-02.atf01: 172.29.0.72:4648 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.224824 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [WARN] memberlist: Refuting an alive message Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.402430 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.402453 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.402466 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.402477 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.402488 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.402523 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.402535 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.402692 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.402845 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.402863 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.403070 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.403086 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.403182 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.403194 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.403206 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.403217 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.403230 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34 [INFO] serf: Re-joined to previously known node: consulnomad-02.atf01: 172.29.0.72:4648 Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.403346 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.403359 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:28:34 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:34.403370 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:28:35 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:35 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:28:35 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:35 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 49 Aug 23 15:28:35 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:35 [INFO] raft: Election won. Tally: 2 Aug 23 15:28:35 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:35 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:28:35 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:35 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:28:35 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:35 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:28:35 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:35.905303 [INFO] nomad: cluster leadership acquired Aug 23 15:28:35 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:35 [INFO] raft: pipelining replication to peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} Aug 23 15:28:35 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:35 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:36 ip-172-24-20-236 nomad.sh[25863]: 2018/08/23 15:28:36 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: fatal error: concurrent map read and map write Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 113 [running]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc4205999c0 sp=0xc4205999a0 pc=0x42d161 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: runtime.mapaccess2(0x1844be0, 0xc42020db00, 0xc420599a88, 0x412539, 0xc42033a8d8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc420599a08 sp=0xc4205999c0 pc=0x409c75 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc420216520, 0x1d6dd40, 0xc42033a8d8, 0x3051286, 0xed3108ca9, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:94 +0xe2 fp=0xc420599af8 sp=0xc420599a08 pc=0xf20ca2 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4203190e0, 0xc4209eda20, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc420599b60 sp=0xc420599af8 pc=0xf5388e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4203190e0, 0xc4209eda20) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc420599b98 sp=0xc420599b60 pc=0xf534b3 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4204a0c80, 0xc4204d2001, 0xc4204d20c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc420599c88 sp=0xc420599b98 pc=0xf77602 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4204a0c80, 0xc420e455c0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc420599d38 sp=0xc420599c88 pc=0xf76fff Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4204a0c80, 0xc420e455c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc420599fa0 sp=0xc420599d38 pc=0xf76b90 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc4214420e0, 0xc4204a0c80, 0xc420e455c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc420599fc8 sp=0xc420599fa0 pc=0xfb8e9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: runtime.goexit() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420599fd0 sp=0xc420599fc8 pc=0x45c711 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 1 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420356d90, 0xc42049d400) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420356d90, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc42047ef00, 0xc42047ef00, 0xc4201f1d20, 0x37) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204ddf68) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc42009e058) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: main.main() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 5 [syscall]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: os/signal.signal_recv(0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: os/signal.loop() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by os/signal.init.0 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 38 [IO wait]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.runtime_pollWait(0x7f131b294f00, 0x72, 0xc4205969a8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).wait(0xc420456898, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).waitRead(0xc420456898, 0xc4205a0000, 0x1000, 0x1000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*FD).Read(0xc420456880, 0xc4205a0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*netFD).Read(0xc420456880, 0xc4205a0000, 0x1000, 0x1000, 0x458600, 0xc420092f00, 0x4) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*conn).Read(0xc42000e5b8, 0xc4205a0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/http.(*persistConn).Read(0xc4200da5a0, 0xc4205a0000, 0x1000, 0x1000, 0xc420596b98, 0x405675, 0xc42009e240) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).fill(0xc420086480) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).Peek(0xc420086480, 0x1, 0x0, 0x0, 0x1, 0xc420eec540, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/http.(*persistConn).readLoop(0xc4200da5a0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by net/http.(*Transport).dialConn Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 191 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420f715e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 169 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4211ac8c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 39 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/http.(*persistConn).writeLoop(0xc4200da5a0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by net/http.(*Transport).dialConn Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 68 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc420405b80, 0x1d71840, 0xc42020f1c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 11 [select, locked to thread]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: runtime.selectgo(0xc420055f50, 0xc4202f66c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: runtime.ensureSigM.func1() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: runtime.goexit() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 12 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc420405ac0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 13 [sleep]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: time.Sleep(0x3b9aca00) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc4202f51e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 14 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc42023e480) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 15 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc42038d6e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 131 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc4210b4630) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 66 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc420388a90) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0xc420204090, 0xc4202040f0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc42020df50, 0xc420204890) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 166 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202f5290) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 138 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4204f2000, 0xc4202926c0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4204f2000, 0xc4202926c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204f2000, 0xc4204d3940) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 165 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202f5290) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 164 [IO wait]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.runtime_pollWait(0x7f131b294a20, 0x72, 0xc420634ce8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).wait(0xc420be4b18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).waitRead(0xc420be4b18, 0xc420eca000, 0x1000, 0x1000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*FD).Read(0xc420be4b00, 0xc420eca000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*netFD).Read(0xc420be4b00, 0xc420eca000, 0x1000, 0x1000, 0xdd, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*conn).Read(0xc4202181b8, 0xc420eca000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).Read(0xc4210c1740, 0xc42117aa40, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: io.ReadAtLeast(0x1d59aa0, 0xc4210c1740, 0xc42117aa40, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: io.ReadFull(0x1d59aa0, 0xc4210c1740, 0xc42117aa40, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202f5290, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202f5290) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 167 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42023e0c0, 0xc420ecb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).fill(0xc420a8ade0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).ReadByte(0xc420a8ade0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4210d5b00, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4210c24a0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420a8ae40, 0x16e4b60, 0xc4210c2500) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420a8ae40, 0x16e4b60, 0xc4210c2500, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4211ac200, 0x16e4b60, 0xc4210c2500, 0xc4210c2500, 0xc421332708) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4211ac200, 0xc4210c2500, 0x2d, 0xc422eda100) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequestHeader(0xc4213326e0, 0x1d72340, 0xc4211ac200, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4203d59a0, 0x8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequest(0xc4213326e0, 0x1d72340, 0xc4211ac200, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).ServeRequest(0xc4213326e0, 0x1d72340, 0xc4211ac200, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d0e0, 0xc42023e0c0, 0xc4213326e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 168 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42023e600, 0xc4201bf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).fill(0xc4203f5b00) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).ReadByte(0xc4203f5b00, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:37 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420c4a360, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421331000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4203f5b60, 0x16e4b60, 0xc421331060) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4203f5b60, 0x16e4b60, 0xc421331060, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4211ac740, 0x16e4b60, 0xc421331060, 0xc421331060, 0xc421332708) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4211ac740, 0xc421331060, 0x2d, 0xc420d4b520) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequestHeader(0xc4213326e0, 0x1d72340, 0xc4211ac740, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4203d59a0, 0x8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequest(0xc4213326e0, 0x1d72340, 0xc4211ac740, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).ServeRequest(0xc4213326e0, 0x1d72340, 0xc4211ac740, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:37 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d0e0, 0xc42023e600, 0xc4213326e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 163 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202f5290, 0xc420638d10, 0x30, 0xf9866a) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202f5290, 0x1c61988, 0xc4204a0c80, 0x1d71840, 0xc4210b01c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d320, 0xc4202181b8, 0xc421332640) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d320, 0xc4202181b8, 0xc421332640) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 28 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4203f2000, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc420244160) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 133 [IO wait]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.runtime_pollWait(0x7f131b294d60, 0x72, 0x0) Aug 23 15:28:37 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).wait(0xc420456918, 0x72, 0xc42020e200, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).waitRead(0xc420456918, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*FD).Accept(0xc420456900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*netFD).accept(0xc420456900, 0xc42033a0f0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*TCPListener).accept(0xc42000e638, 0x458e40, 0xc4204b7f90, 0xc4204b7f98) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*TCPListener).AcceptTCP(0xc42000e638, 0x1c63658, 0xc4207ac3c0, 0xc42033a0f0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4207ac3c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 112 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420b18180, 0xc420a4f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).fill(0xc420a8b1a0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).ReadByte(0xc420a8b1a0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a2cfc0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4210c2740) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420a8b200, 0x16e4b60, 0xc4210c27a0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420a8b200, 0x16e4b60, 0xc4210c27a0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4208afe00, 0x16e4b60, 0xc4210c27a0, 0xc4210c27a0, 0xc421332708) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4208afe00, 0xc4210c27a0, 0x2d, 0xc422eda340) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequestHeader(0xc4213326e0, 0x1d72340, 0xc4208afe00, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4203d59a0, 0x8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequest(0xc4213326e0, 0x1d72340, 0xc4208afe00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).ServeRequest(0xc4213326e0, 0x1d72340, 0xc4208afe00, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d0e0, 0xc420b18180, 0xc4213326e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 135 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4207ac3c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 48 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4204f2000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4204f2000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4204f2000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204f2000, 0xc4212863e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 139 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420a70b80) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 130 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4204f2000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204f2000, 0xc421286400) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 132 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420f46680) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 134 [IO wait]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.runtime_pollWait(0x7f131b294c90, 0x72, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).wait(0xc420456998, 0x72, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).waitRead(0xc420456998, 0xc422994000, 0x10000, 0x10000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*FD).ReadFrom(0xc420456980, 0xc422994000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*netFD).readFrom(0xc420456980, 0xc422994000, 0x10000, 0x10000, 0x28, 0x28, 0x1d62b40, 0x412539, 0xc420e6ce40) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*UDPConn).readFrom(0xc42000e650, 0xc422994000, 0x10000, 0x10000, 0x101, 0xc420e6ce18, 0xc420e6ce90, 0x443cf7) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*UDPConn).ReadFrom(0xc42000e650, 0xc422994000, 0x10000, 0x10000, 0x10000, 0x10000, 0x5f5c, 0xb5f27fb4, 0x2b96de0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4207ac3c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 115 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4207ac3c0, 0x12a05f200, 0xc4210acba0, 0xc420f69380, 0xc421286550) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:110 +0xef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 49 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4204f2000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204f2000, 0xc4212863f0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 116 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4207ac3c0, 0xc420f69380) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 117 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4207ac3c0, 0x1dcd6500, 0xc4210acc00, 0xc420f69380, 0xc421286560) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 118 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc420569800) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 119 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc420569800) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 120 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420569800, 0x1bdd09a, 0x6, 0xc4210b4780) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 121 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420569800, 0x1bdba6c, 0x5, 0xc4210b47b0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 122 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420569800, 0x1bdbc6a, 0x5, 0xc4210b47e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 156 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202448f0, 0xc420e70d10, 0x30, 0xf9866a) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202448f0, 0x1c61988, 0xc4204a0c80, 0x1d71840, 0xc4210b01c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d320, 0xc42000f308, 0xc42021d0e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d320, 0xc42000f308, 0xc42021d0e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 124 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4210acc60, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4210acc60, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4210acc60) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 125 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4210accc0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4210accc0, 0x1dcd6500, 0x1bd9928, 0x2, 0x1bdab30, 0x4, 0xc420eda400) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4210accc0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 126 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4210acd20, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4210acd20, 0x1dcd6500, 0xc420d84140, 0x3, 0x4, 0xc4201ba560, 0x15) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4210acd20) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 127 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4210acd80, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4210acd80, 0x1dcd6500, 0xc420e72120, 0x9ebabe, 0x1, 0x0, 0x435248) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4210acd80) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 128 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc421444180, 0xc4204a0c80) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 129 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4204a0c80) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 146 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4204a0c80) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 147 [IO wait]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.runtime_pollWait(0x7f131b294e30, 0x72, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).wait(0xc420142218, 0x72, 0xc42020f300, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).waitRead(0xc420142218, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*FD).Accept(0xc420142200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*netFD).accept(0xc420142200, 0x1, 0x1, 0xc420e6ff20) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*TCPListener).accept(0xc42000e430, 0x406b4a, 0xc420f10f60, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*TCPListener).Accept(0xc42000e430, 0xc420f10f60, 0x0, 0x3f800000, 0xc4210b01c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4204a0c80, 0x1d71840, 0xc4210b01c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 148 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4203190e0, 0x3b9aca00, 0xc4202f6b40) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 149 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4203d5c70, 0x3b9aca00, 0xc4202f6b40) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 150 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420386ab0, 0x3b9aca00, 0xc4202f6b40) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 151 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc420388a90, 0x3b9aca00, 0xc4202f6b40) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 152 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4204a0c80) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 153 [IO wait]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.runtime_pollWait(0x7f131b294bc0, 0x72, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).wait(0xc420d70b98, 0x72, 0xc42020f400, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).waitRead(0xc420d70b98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*FD).Accept(0xc420d70b80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*netFD).accept(0xc420d70b80, 0x402037, 0xc4203184e0, 0x1805ec0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*TCPListener).accept(0xc42000f060, 0x458e40, 0xc42063be90, 0xc42063be98) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*TCPListener).Accept(0xc42000f060, 0x1c65af8, 0xc420318460, 0x1d71900, 0xc4210d45d0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/http.(*Server).Serve(0xc420f8b1e0, 0x1d6fc00, 0xc42000f060, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/http.Serve(0x1d6fc00, 0xc42000f060, 0x1d5df00, 0xc4210d4510, 0x4, 0xc420a3f900) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc4210b0280, 0x1d6fc00, 0xc42000f060, 0xc4212869f0, 0xc421444240) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 154 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420d70c00, 0xc421286a00, 0xc420f02120) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 155 [sleep]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: time.Sleep(0x3b2e907cb) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc420356d90, 0xc420d70c00) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 157 [IO wait]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.runtime_pollWait(0x7f131b294af0, 0x72, 0xc420593ce8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).wait(0xc4207e2998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).waitRead(0xc4207e2998, 0xc420f80000, 0x1000, 0x1000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*FD).Read(0xc4207e2980, 0xc420f80000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*netFD).Read(0xc4207e2980, 0xc420f80000, 0x1000, 0x1000, 0x8e, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*conn).Read(0xc42000f308, 0xc420f80000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).Read(0xc42018dce0, 0xc420d38a94, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: io.ReadAtLeast(0x1d59aa0, 0xc42018dce0, 0xc420d38a94, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: io.ReadFull(0x1d59aa0, 0xc42018dce0, 0xc420d38a94, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202448f0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202448f0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 158 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202448f0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 159 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202448f0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 160 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420b180c0, 0xc420f81000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).fill(0xc420406600) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).ReadByte(0xc420406600, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420ab7740, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4204d2f60) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4204066c0, 0x16e4b60, 0xc4204d2fc0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4204066c0, 0x16e4b60, 0xc4204d2fc0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4210b1a80, 0x16e4b60, 0xc4204d2fc0, 0xc4204d2fc0, 0xc42021d1f8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4210b1a80, 0xc4204d2fc0, 0x2d, 0xc42209ce60) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequestHeader(0xc42021d1d0, 0x1d72340, 0xc4210b1a80, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4203d59a0, 0x8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequest(0xc42021d1d0, 0x1d72340, 0xc4210b1a80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).ServeRequest(0xc42021d1d0, 0x1d72340, 0xc4210b1a80, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d0e0, 0xc420b180c0, 0xc42021d1d0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 161 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4205bc300, 0xc420b8e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).fill(0xc42132d500) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).ReadByte(0xc42132d500, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420c80540, 0x42c30b) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421108d60) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42132d560, 0x16e4b60, 0xc421108dc0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42132d560, 0x16e4b60, 0xc421108dc0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420c790c0, 0x16e4b60, 0xc421108dc0, 0xc421108dc0, 0xc42021d1f8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420c790c0, 0xc421108dc0, 0x2d, 0xc422eda5a0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequestHeader(0xc42021d1d0, 0x1d72340, 0xc420c790c0, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequest(0xc42021d1d0, 0x1d72340, 0xc420c790c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).ServeRequest(0xc42021d1d0, 0x1d72340, 0xc420c790c0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d0e0, 0xc4205bc300, 0xc42021d1d0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 178 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202906e0, 0xc42063ad10, 0x30, 0xf9866a) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202906e0, 0x1c61988, 0xc4204a0c80, 0x1d71840, 0xc4210b01c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d320, 0xc420218880, 0xc420d82c80) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d320, 0xc420218880, 0xc420d82c80) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 179 [IO wait]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.runtime_pollWait(0x7f131b294950, 0x72, 0xc420edfce8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).wait(0xc4203ea698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).waitRead(0xc4203ea698, 0xc42095e000, 0x1000, 0x1000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*FD).Read(0xc4203ea680, 0xc42095e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*netFD).Read(0xc4203ea680, 0xc42095e000, 0x1000, 0x1000, 0xc4204b65d8, 0x1, 0xc4204b6608) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*conn).Read(0xc420218880, 0xc42095e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).Read(0xc42132d920, 0xc420d39b20, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: io.ReadAtLeast(0x1d59aa0, 0xc42132d920, 0xc420d39b20, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: io.ReadFull(0x1d59aa0, 0xc42132d920, 0xc420d39b20, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202906e0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202906e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 180 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202906e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 181 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202906e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 182 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42023e6c0, 0xc42095f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).fill(0xc4210d2e40) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).ReadByte(0xc4210d2e40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420bd05a0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421108f60) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210d2ea0, 0x16e4b60, 0xc421108fc0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210d2ea0, 0x16e4b60, 0xc421108fc0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420b93ac0, 0x16e4b60, 0xc421108fc0, 0xc421108fc0, 0xc420d82d48) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420b93ac0, 0xc421108fc0, 0x2d, 0xc42085e460) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequestHeader(0xc420d82d20, 0x1d72340, 0xc420b93ac0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4203d59a0, 0x8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequest(0xc420d82d20, 0x1d72340, 0xc420b93ac0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).ServeRequest(0xc420d82d20, 0x1d72340, 0xc420b93ac0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d0e0, 0xc42023e6c0, 0xc420d82d20) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 137 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4204f2000, 0xc420292600, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4204f2000, 0xc420292600) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204f2000, 0xc4204d3920) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 211 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4205bc0c0, 0xc420f6c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).fill(0xc4208b9d40) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).ReadByte(0xc4208b9d40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420f35530, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206c8840) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4208b9da0, 0x16e4b60, 0xc4206c88a0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4208b9da0, 0x16e4b60, 0xc4206c88a0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4215ea100, 0x16e4b60, 0xc4206c88a0, 0xc4206c88a0, 0xc421332708) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4215ea100, 0xc4206c88a0, 0x2d, 0xc420b50940) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequestHeader(0xc4213326e0, 0x1d72340, 0xc4215ea100, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4203d59a0, 0x8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequest(0xc4213326e0, 0x1d72340, 0xc4215ea100, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).ServeRequest(0xc4213326e0, 0x1d72340, 0xc4215ea100, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d0e0, 0xc4205bc0c0, 0xc4213326e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 186 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4204f2000, 0xc420292600, 0xc420defd40) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204f2000, 0xc4211095a0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 187 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4204f2000, 0xc4202926c0, 0xc420defec0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204f2000, 0xc4211096e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 170 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4204f2000, 0xc420292600, 0x1d6eb80, 0xc4211ac8c0, 0xc420e01680, 0xc420e01740) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204f2000, 0xc4211ac900) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 140 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4204f2000, 0xc4202926c0, 0x1d6eb80, 0xc420a70b80, 0xc4207fbb00, 0xc4207fbb60) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204f2000, 0xc420a70e00) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 171 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42023e000, 0xc420eeb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).fill(0xc4210ad5c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).ReadByte(0xc4210ad5c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a3b1d0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420abeac0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210ad620, 0x16e4b60, 0xc420abeb20) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210ad620, 0x16e4b60, 0xc420abeb20, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4210b0d00, 0x16e4b60, 0xc420abeb20, 0xc420abeb20, 0xc42021d1f8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4210b0d00, 0xc420abeb20, 0x2d, 0xc420b504c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequestHeader(0xc42021d1d0, 0x1d72340, 0xc4210b0d00, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4203d59a0, 0x8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).readRequest(0xc42021d1d0, 0x1d72340, 0xc4210b0d00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).ServeRequest(0xc42021d1d0, 0x1d72340, 0xc4210b0d00, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d0e0, 0xc42023e000, 0xc42021d1d0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 172 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200cc2c0, 0xc420066d10, 0x30, 0xf9866a) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200cc2c0, 0x1c61988, 0xc4204a0c80, 0x1d71840, 0xc4210b01c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d320, 0xc42022a008, 0xc421514640) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d320, 0xc42022a008, 0xc421514640) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 173 [IO wait]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.runtime_pollWait(0x7f131b294610, 0x72, 0xc420dbdce8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).wait(0xc420457318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).waitRead(0xc420457318, 0xc420bbf000, 0x1000, 0x1000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*FD).Read(0xc420457300, 0xc420bbf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*netFD).Read(0xc420457300, 0xc420bbf000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*conn).Read(0xc42022a008, 0xc420bbf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).Read(0xc42146af60, 0xc420778010, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: io.ReadAtLeast(0x1d59aa0, 0xc42146af60, 0xc420778010, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: io.ReadFull(0x1d59aa0, 0xc42146af60, 0xc420778010, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200cc2c0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200cc2c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 174 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200cc2c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 175 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200cc2c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 176 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420407a40, 0xc420b7b700, 0x20, 0x20, 0x169d4e0, 0xc420b9e000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421831590, 0x1d718c0, 0xc420407a40, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4204a0c80, 0xc420b7b950, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420407380, 0xc4207db800, 0xc420967e00, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: reflect.Value.call(0xc4205899e0, 0xc42022af28, 0x13, 0x1bda9a0, 0x4, 0xc420b7bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: reflect.Value.Call(0xc4205899e0, 0xc42022af28, 0x13, 0xc420b7bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*service).call(0xc4212cb200, 0xc421332af0, 0xc4207549c0, 0x0, 0xc4207db400, 0xc420ebb1c0, 0x19758e0, 0xc4207db800, 0x16, 0x169e020, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).ServeRequest(0xc421332af0, 0x1d72340, 0xc4212cb240, 0x0, 0xc420b18240) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d0e0, 0xc420b18240, 0xc421332af0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 190 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420f715e0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 192 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4203d5c70, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4204a0c80) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 197 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420794000, 0xc420eddd10, 0x30, 0xf9866a) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420794000, 0x1c61988, 0xc4204a0c80, 0x1d71840, 0xc4210b01c0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d320, 0xc42022a1a8, 0xc4214f2870) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d320, 0xc42022a1a8, 0xc4214f2870) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 198 [IO wait]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.runtime_pollWait(0x7f131b294470, 0x72, 0xc4209efce8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).wait(0xc420d70218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*pollDesc).waitRead(0xc420d70218, 0xc420f4e000, 0x1000, 0x1000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: internal/poll.(*FD).Read(0xc420d70200, 0xc420f4e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*netFD).Read(0xc420d70200, 0xc420f4e000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net.(*conn).Read(0xc42022a1a8, 0xc420f4e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: bufio.(*Reader).Read(0xc421239560, 0xc4205dfe10, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: io.ReadAtLeast(0x1d59aa0, 0xc421239560, 0xc4205dfe10, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: io.ReadFull(0x1d59aa0, 0xc421239560, 0xc4205dfe10, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420794000, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420794000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 199 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420794000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 200 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420794000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 201 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420a60780, 0xc420b77700, 0x20, 0x20, 0x169d4e0, 0xc420b9e000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4217fdc80, 0x1d718c0, 0xc420a60780, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4204a0c80, 0xc420b77950, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4211db4a0, 0xc420a80180, 0xc420bfac00, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: reflect.Value.call(0xc4205899e0, 0xc42022b350, 0x13, 0x1bda9a0, 0x4, 0xc420b77d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: reflect.Value.Call(0xc4205899e0, 0xc42022b350, 0x13, 0xc420b77d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*service).call(0xc421439340, 0xc4214f2910, 0xc420779b08, 0x0, 0xc420be4800, 0xc4204d3220, 0x19758e0, 0xc420a80180, 0x16, 0x169e020, ...) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: net/rpc.(*Server).ServeRequest(0xc4214f2910, 0x1d72340, 0xc421439380, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4204a0c80, 0x1d71840, 0xc4210b01c0, 0x1d7d0e0, 0xc42023e180, 0xc4214f2910) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 193 [semacquire]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: sync.runtime_SemacquireMutex(0xc42031916c, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: sync.(*Mutex).Lock(0xc420319168) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: sync.(*RWMutex).Lock(0xc420319168) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4203190e0, 0x1d71840, 0xc422f15300) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 226 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420386ab0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 227 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420386ab0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 228 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc422a55cb0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 229 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422f15340, 0xc420dbfce8, 0x20, 0x20, 0x18, 0xc422fce260) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4229de210, 0x1d71840, 0xc422f15340, 0xc422fce260, 0x184987) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204d20c0, 0xc4231ebed0, 0x184987, 0x1d71840, 0xc422f15340, 0xc4201427e0, 0xc422a55c80, 0xc420c43530, 0x24, 0x2bba400) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420142780, 0x1d71840, 0xc422f15340, 0x184987, 0xc422eec040, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420142780, 0x1d71840, 0xc422f15340) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 230 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422f153c0, 0xc4210cd770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4229de360, 0x1d71840, 0xc422f153c0, 0x0, 0x1849b5) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204d20c0, 0xc4210cd960, 0x1849b5, 0x1d71840, 0xc422f153c0, 0x552bde, 0xc4210cd950, 0xc4210cd970, 0xf33ba5, 0xc422019f68) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc422019f10, 0x1d71840, 0xc422f153c0, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc422019f10) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 231 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422f15380, 0xc4231e7a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422cbc1e0, 0x1d71840, 0xc422f15380, 0xc422cbc2d0, 0x1833a4) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204d20c0, 0xc4231e7c68, 0x1833a4, 0x1d71840, 0xc422f15380, 0x552bde, 0xc4203d5a40, 0x2, 0xc420f44100, 0x40) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422f15400, 0x1833a4, 0x3b, 0xc4209f2de8, 0x1, 0x1) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422f15400) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 142 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4229de000, 0xc4227e8000) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 143 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4229de000, 0xc4227e8060) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 232 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422f15440) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 233 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4207ac5a0, 0x1d71840, 0xc422f15380) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 144 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc421fc0140) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: goroutine 213 [select]: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422eec100, 0xc420dc1d28, 0x20, 0x20, 0xc420088000, 0x1bd9a98) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422cbc1b0, 0x1d71840, 0xc422eec100, 0xc4227c02a0, 0x1849b5) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4204d20c0, 0xc4231e9f20, 0x1849b5, 0x1d71840, 0xc422eec100, 0x0, 0xc422ad1e00, 0xf2c8ca, 0x1, 0x0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc421fc0140, 0x1849b5, 0xc422ad1e60, 0xf2c920, 0x1, 0x0, 0x0, 0xc4212f0eb8) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc421fc0140, 0x1849b5, 0xc4227982a0) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25863]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:28:37 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:28:37 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:28:37 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: ==> Starting Nomad agent... Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: ==> Nomad agent configuration: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: Client: false Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: Log Level: INFO Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: Region: amrv01 (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: Server: true Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: Version: 0.8.4 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.735515 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: Attempting re-join to previously known node: consulnomad-03.aaf01: 172.26.1.78:4648 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.735639 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.915988 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916011 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [WARN] memberlist: Refuting an alive message Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916022 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916169 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916184 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916256 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916287 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916300 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916343 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916764 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916958 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916978 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.916990 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.917001 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.917014 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.917030 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.917041 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.917051 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.917065 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37 [INFO] serf: Re-joined to previously known node: consulnomad-03.aaf01: 172.26.1.78:4648 Aug 23 15:28:37 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:37.917213 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:39 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:39 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:40 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:40 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:28:40 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:40 [WARN] raft: Rejecting vote request from 172.24.0.121:4647 since we have a leader: 172.24.10.89:4647 Aug 23 15:28:40 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:40 [WARN] raft: Failed to get previous log: 1598279 log not found (last: 1598277) Aug 23 15:28:40 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:40.829681 [ERR] http: Request /v1/agent/health?type=server, error: {"server":{"ok":false,"message":"rpc error: rpc error: EOF"}} Aug 23 15:28:42 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:42 [INFO] serf: EventMemberUpdate: ip-172-24-0-121.amrv01 Aug 23 15:28:42 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:42 [WARN] raft: Rejecting vote request from 172.24.10.89:4647 since we have a leader: 172.24.0.121:4647 Aug 23 15:28:43 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:43.818438 [INFO] server.nomad: successfully contacted 20 Nomad Servers Aug 23 15:28:44 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:44 [WARN] raft: Heartbeat timeout from "172.24.0.121:4647" reached, starting election Aug 23 15:28:44 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:44 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 54 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:45 [INFO] raft: Duplicate RequestVote for same term: 54 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:45 [INFO] raft: Election won. Tally: 2 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:45 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:45 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:45 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:45.053086 [INFO] nomad: cluster leadership acquired Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: 2018/08/23 15:28:45 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: fatal error: concurrent map writes Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 356 [running]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc422a13978 sp=0xc422a13958 pc=0x42d161 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: runtime.mapassign(0x1844be0, 0xc42037dc50, 0xc422a13aa8, 0x2bba400) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/hashmap.go:519 +0x56e fp=0xc422a13a08 sp=0xc422a13978 pc=0x40a41e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc4202c78e0, 0x1d6dd40, 0xc42020da20, 0x386bc49b, 0xed310848b, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:99 +0x1d1 fp=0xc422a13af8 sp=0xc422a13a08 pc=0xf20d91 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4200d40a0, 0xc420966b00, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc422a13b60 sp=0xc422a13af8 pc=0xf5388e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4200d40a0, 0xc420966b00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc422a13b98 sp=0xc422a13b60 pc=0xf534b3 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4200ee280, 0xc42059bc01, 0xc42059bca0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc422a13c88 sp=0xc422a13b98 pc=0xf77602 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4200ee280, 0xc42309d440, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc422a13d38 sp=0xc422a13c88 pc=0xf76fff Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4200ee280, 0xc42309d440) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc422a13fa0 sp=0xc422a13d38 pc=0xf76b90 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420c54000, 0xc4200ee280, 0xc42309d440) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc422a13fc8 sp=0xc422a13fa0 pc=0xfb8e9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: runtime.goexit() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc422a13fd0 sp=0xc422a13fc8 pc=0x45c711 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 1 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42031c930, 0xc420497040) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42031c930, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc420316dc0, 0xc420316dc0, 0xc420289540, 0x37) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4201cdf68) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc42009e058) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: main.main() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 5 [syscall]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: os/signal.signal_recv(0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: os/signal.loop() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by os/signal.init.0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 194 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4204e69c0, 0xc42119b700, 0x20, 0x20, 0x169d4e0, 0xc4205f6e20) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420c73c80, 0x1d718c0, 0xc4204e69c0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc42119b950, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42111e060, 0xc4208d5d80, 0xc420fd7590, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.call(0xc4205e4720, 0xc4205a0a80, 0x13, 0x1bda9a0, 0x4, 0xc42119bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.Call(0xc4205e4720, 0xc4205a0a80, 0x13, 0xc42119bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*service).call(0xc420c57700, 0xc42062cfa0, 0xc420c02fd8, 0x0, 0xc420af0080, 0xc420b9dee0, 0x19758e0, 0xc4208d5d80, 0x16, 0x169e020, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc42062cfa0, 0x1d72340, 0xc420c57740, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc420084600, 0xc42062cfa0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 49 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420362b00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 48 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420362b00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 25 [sleep]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: time.Sleep(0x3b9aca00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc42053a000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 63 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420cf83c0, 0xc420ee9140) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 24 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42042b9c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 37 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420217590, 0x1c636a0, 0xc4203628f0, 0x1d71840, 0xc4205fa400) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4203628f0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 50 [select, locked to thread]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: runtime.selectgo(0xc420050750, 0xc4202ee960) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: runtime.ensureSigM.func1() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: runtime.goexit() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 26 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420176300) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 27 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc420188480) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 166 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*persistConn).writeLoop(0xc420cfc000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by net/http.(*Transport).dialConn Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 29 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4202251e0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4205800c0, 0xc420366370) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 73 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc420213280, 0x1d71840, 0xc420582000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 177 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd3a0, 0x72, 0xc4210face8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc42021c018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc42021c018, 0xc420ff3000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc42021c000, 0xc420ff3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc42021c000, 0xc420ff3000, 0x1000, 0x1000, 0xc4210fadd8, 0x0, 0xc4210fae08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc4205a0018, 0xc420ff3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc4200861e0, 0xc4210e7430, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc4200861e0, 0xc4210e7430, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc4200861e0, 0xc4210e7430, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200cc2c0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200cc2c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 206 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42053a420, 0xc422a15d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42053a420, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a0348, 0xc422fa2280) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a0348, 0xc422fa2280) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 71 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bdf00, 0x72, 0xc4200659a8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420142298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420142298, 0xc4205d8000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420142280, 0xc4205d8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420142280, 0xc4205d8000, 0x1000, 0x1000, 0x458600, 0xc421305c80, 0x4) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc4205a0028, 0xc4205d8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*persistConn).Read(0xc4205ae120, 0xc4205d8000, 0x1000, 0x1000, 0xc420065b98, 0x405675, 0xc420586480) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc420592780) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Peek(0xc420592780, 0x1, 0x0, 0x0, 0x1, 0xc420d7ec00, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*persistConn).readLoop(0xc4205ae120) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by net/http.(*Transport).dialConn Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 72 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*persistConn).writeLoop(0xc4205ae120) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by net/http.(*Transport).dialConn Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 47 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd950, 0x72, 0xc42111bce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420bac518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420bac518, 0xc4207c9000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420bac500, 0xc4207c9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420bac500, 0xc4207c9000, 0x1000, 0x1000, 0xc42111bdd8, 0x1, 0xc42111be08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc4205a0778, 0xc4207c9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc4201891a0, 0xc420b309b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc4201891a0, 0xc420b309b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc4201891a0, 0xc420b309b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420362b00, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420362b00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 65 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc421304600) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 129 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42053a160, 0xc420554d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42053a160, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc420218410, 0xc42051d720) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc420218410, 0xc42051d720) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 165 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bda20, 0x72, 0xc420ddb9a8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc42044f498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc42044f498, 0xc420a6e000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc42044f480, 0xc420a6e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc42044f480, 0xc420a6e000, 0x1000, 0x1000, 0x458600, 0xc420092900, 0x4) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc42020c1d8, 0xc420a6e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*persistConn).Read(0xc420cfc000, 0xc420a6e000, 0x1000, 0x1000, 0xc420ddbb98, 0x405675, 0xc420f29c20) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc420fec8a0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Peek(0xc420fec8a0, 0x1, 0x0, 0x0, 0x1, 0xc420efdc20, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*persistConn).readLoop(0xc420cfc000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by net/http.(*Transport).dialConn Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 243 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200cc2c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 46 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420362b00, 0xc420de1d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420362b00, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a0778, 0xc42062cf00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a0778, 0xc42062cf00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 242 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200cc2c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 189 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42053a2c0, 0xc421116d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42053a2c0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a0c20, 0xc4205aa320) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a0c20, 0xc4205aa320) Aug 23 15:28:45 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 64 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420cf83c0, 0x1dcd6500, 0xc420ff5bc0, 0xc420ee9140, 0xc42056e5d0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 213 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd2d0, 0x72, 0xc4210f9ce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420f58e98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420f58e98, 0xc421288000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420f58e80, 0xc421288000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420f58e80, 0xc421288000, 0x1000, 0x1000, 0xc4210f9dd8, 0x0, 0xc4210f9e08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc42020c020, 0xc421288000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc420f50060, 0xc420c54d90, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc420f50060, 0xc420c54d90, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc420f50060, 0xc420c54d90, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42053a210, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42053a210) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 121 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4204ea000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4204ea000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4204ea000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204ea000, 0xc420f09510) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 62 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420cf83c0, 0x12a05f200, 0xc420ff5b60, 0xc420ee9140, 0xc42056e5c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 122 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4204ea000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204ea000, 0xc420f09590) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 123 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4204ea000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204ea000, 0xc420f095a0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 124 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420fd1ec0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 125 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4205ee1a0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 126 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bdd60, 0x72, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420e8a018, 0x72, 0xc420c56000, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420e8a018, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Accept(0xc420e8a000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).accept(0xc420e8a000, 0xc422a19050, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*TCPListener).accept(0xc420218138, 0x458e40, 0xc4204b5f90, 0xc4204b5f98) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*TCPListener).AcceptTCP(0xc420218138, 0x1c63658, 0xc420cf83c0, 0xc422a19050) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420cf83c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 127 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bdc90, 0x72, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420e8a098, 0x72, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420e8a098, 0xc423800000, 0x10000, 0x10000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).ReadFrom(0xc420e8a080, 0xc423800000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).readFrom(0xc420e8a080, 0xc423800000, 0x10000, 0x10000, 0x29, 0x29, 0x1d62b40, 0x412539, 0xc421091e40) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*UDPConn).readFrom(0xc420218160, 0xc423800000, 0x10000, 0x10000, 0x101, 0xc421091e18, 0xc421091e90, 0x443cf7) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*UDPConn).ReadFrom(0xc420218160, 0xc423800000, 0x10000, 0x10000, 0x10000, 0x10000, 0xd4d8, 0x1b96f9284, 0x2b96de0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420cf83c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 128 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420cf83c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 130 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc421304600) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 131 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421304600, 0x1bdd09a, 0x6, 0xc421000030) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 132 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421304600, 0x1bdba6c, 0x5, 0xc421000060) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 133 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421304600, 0x1bdbc6a, 0x5, 0xc421000090) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 345 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4225f4fd0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 135 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420ff5c20, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ff5c20, 0x1dcd6500, 0x0, 0x0, 0x0, 0xc4205861e0, 0x71a2da) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ff5c20) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 136 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420ff5c80, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ff5c80, 0x1dcd6500, 0x63c900, 0x1, 0x0, 0x435248, 0x1c66060) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ff5c80) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 137 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420ff5ce0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ff5ce0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ff5ce0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 138 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420ff5d40, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ff5d40, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ff5d40) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 139 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420ff7ec0, 0xc4200ee280) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 140 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4200ee280) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 141 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4200ee280) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 142 [runnable]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: syscall.Syscall6(0x36, 0x23, 0x6, 0x1, 0xc420553de4, 0x4, 0x0, 0x0, 0x1, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: syscall.setsockopt(0x23, 0x6, 0x1, 0xc420553de4, 0x4, 0xc420553e08, 0x18) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/syscall/zsyscall_linux_amd64.go:1608 +0x7c Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: syscall.SetsockoptInt(0x23, 0x6, 0x1, 0x1, 0x7f7dd111a458, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/syscall/syscall_unix.go:275 +0x5b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).SetsockoptInt(0xc42227a980, 0x6, 0x1, 0x1, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/sockopt.go:17 +0xa0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.setNoDelay(0xc42227a980, 0xc42000fc01, 0xc42053a000, 0x1d62b00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/tcpsockopt_posix.go:15 +0x50 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.newTCPConn(0xc42227a980, 0xc42227a980) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/tcpsock.go:195 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*TCPListener).accept(0xc42020ce38, 0x406b4a, 0xc420f073e0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/tcpsock_posix.go:140 +0x68 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*TCPListener).Accept(0xc42020ce38, 0xc420f073e0, 0x0, 0x3f800000, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 143 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4200d40a0, 0x3b9aca00, 0xc420376420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 144 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc420217360, 0x3b9aca00, 0xc420376420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 145 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc42053e120, 0x3b9aca00, 0xc420376420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 146 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4202251e0, 0x3b9aca00, 0xc420376420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 147 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4200ee280) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 148 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bdbc0, 0x72, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420e8ae98, 0x72, 0xc420202e00, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420e8ae98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Accept(0xc420e8ae80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).accept(0xc420e8ae80, 0x402037, 0xc421748120, 0x1805ec0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*TCPListener).accept(0xc42000e7c8, 0x458e40, 0xc420060e90, 0xc420060e98) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*TCPListener).Accept(0xc42000e7c8, 0x1c65af8, 0xc4217480a0, 0x1d71900, 0xc420bb6330) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*Server).Serve(0xc420640dd0, 0x1d6fc00, 0xc42000e7c8, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.Serve(0x1d6fc00, 0xc42000e7c8, 0x1d5df00, 0xc420bb6270, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc421298300, 0x1d6fc00, 0xc42000e7c8, 0xc42056eb30, 0xc420ff7f80) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 149 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420e8af00, 0xc42056eb40, 0xc420efc3c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 343 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd105ee58, 0x72, 0xc4210f8ce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc422d2d718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc422d2d718, 0xc421f00000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc422d2d700, 0xc421f00000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc422d2d700, 0xc421f00000, 0x1000, 0x1000, 0xc4210f8dd8, 0x0, 0xc4210f8e08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc422a19650, 0xc421f00000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc420d0eae0, 0xc4218e43e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc420d0eae0, 0xc4218e43e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc420d0eae0, 0xc4218e43e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4225f4fd0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4225f4fd0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 183 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42053a0b0, 0xc42135fd10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42053a0b0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a0038, 0xc421108500) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a0038, 0xc421108500) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 184 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd470, 0x72, 0xc42135ace8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc42097e018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc42097e018, 0xc420a9f000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc42097e000, 0xc420a9f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc42097e000, 0xc420a9f000, 0x1000, 0x1000, 0xc42135add8, 0x1, 0xc42135ae08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc4205a0038, 0xc420a9f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc4210dec60, 0xc420b86000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc4210dec60, 0xc420b86000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc4210dec60, 0xc420b86000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42053a0b0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42053a0b0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 178 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bdaf0, 0x72, 0xc420dddce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420df4e98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420df4e98, 0xc420972000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420df4e80, 0xc420972000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420df4e80, 0xc420972000, 0x1000, 0x1000, 0xc420ddddd8, 0x1, 0xc420ddde08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc420218410, 0xc420972000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc4204e6c00, 0xc421294760, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc4204e6c00, 0xc421294760, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc4204e6c00, 0xc421294760, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42053a160, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42053a160) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 179 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42053a160) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 180 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42053a160) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 219 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4213de000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 239 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420363ce0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 349 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4204ea000, 0xc420a44480, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4204ea000, 0xc420a44480) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204ea000, 0xc4224b7d00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 284 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4213df1e0, 0xc420fe7d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4213df1e0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc422a186a8, 0xc4226be0f0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc422a186a8, 0xc4226be0f0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 294 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42023dd90, 0xc420ddfd10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42023dd90, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc42000f2d0, 0xc4223b8eb0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc42000f2d0, 0xc4223b8eb0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 158 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42023c420, 0xc420ddad10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42023c420, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc42000ebb0, 0xc4212a0d70) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc42000ebb0, 0xc4212a0d70) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 159 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd7b0, 0x72, 0xc420556ce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc4209f7518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc4209f7518, 0xc420d00000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc4209f7500, 0xc420d00000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc4209f7500, 0xc420d00000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc42000ebb0, 0xc420d00000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc420b07e00, 0xc420ae4580, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc420b07e00, 0xc420ae4580, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc420b07e00, 0xc420ae4580, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42023c420, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42023c420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 160 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42023c420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 161 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42023c420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 298 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420a44240, 0xc4213b2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc421a62000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).ReadByte(0xc421a62000, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422a72120, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc423076e80) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421a62060, 0x16e4b60, 0xc423076ee0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421a62060, 0x16e4b60, 0xc423076ee0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4224af380, 0x16e4b60, 0xc423076ee0, 0xc423076ee0, 0xc4223b8f78) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4224af380, 0xc423076ee0, 0x2d, 0xc4227773c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequestHeader(0xc4223b8f50, 0x1d72340, 0xc4224af380, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200b7cc0, 0x8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequest(0xc4223b8f50, 0x1d72340, 0xc4224af380, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc4223b8f50, 0x1d72340, 0xc4224af380, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc420a44240, 0xc4223b8f50) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 197 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4203629a0, 0xc421358d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4203629a0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc420218180, 0xc4213ca870) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc420218180, 0xc4213ca870) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 185 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42053a0b0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 186 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42053a0b0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 187 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204480c0, 0xc420e19000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc420b07860) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).ReadByte(0xc420b07860, 0x101000000000016, 0x0, 0x2d) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42098ed50, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421310840) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420b078c0, 0x16e4b60, 0xc4213108a0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420b078c0, 0x16e4b60, 0xc4213108a0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420f70c80, 0x16e4b60, 0xc4213108a0, 0xc4213108a0, 0xc4211085c8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420f70c80, 0xc4213108a0, 0x2d, 0xc4207a82c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequestHeader(0xc4211085a0, 0x1d72340, 0xc420f70c80, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequest(0xc4211085a0, 0x1d72340, 0xc420f70c80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc4211085a0, 0x1d72340, 0xc420f70c80, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc4204480c0, 0xc4211085a0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 188 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420448480, 0xc420d75000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc4204e6780) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).ReadByte(0xc4204e6780, 0x101000000000016, 0x0, 0x2d) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420fd7110, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420721420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4204e67e0, 0x16e4b60, 0xc420721480) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4204e67e0, 0x16e4b60, 0xc420721480, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420a1b1c0, 0x16e4b60, 0xc420721480, 0xc420721480, 0xc42062cfc8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420a1b1c0, 0xc420721480, 0x2d, 0xc42121ae40) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequestHeader(0xc42062cfa0, 0x1d72340, 0xc420a1b1c0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200b7cc0, 0x8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequest(0xc42062cfa0, 0x1d72340, 0xc420a1b1c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc42062cfa0, 0x1d72340, 0xc420a1b1c0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc420448480, 0xc42062cfa0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 190 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd540, 0x72, 0xc42135dce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc4212ce018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc4212ce018, 0xc4209e9000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc4212ce000, 0xc4209e9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc4212ce000, 0xc4209e9000, 0x1000, 0x1000, 0xc42135ddd8, 0x0, 0xc42135de08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc4205a0c20, 0xc4209e9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc420592f00, 0xc420747340, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc420592f00, 0xc420747340, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc420592f00, 0xc420747340, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42053a2c0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42053a2c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 191 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42053a2c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 192 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42053a2c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 212 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42053a210, 0xc4205e8d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42053a210, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc42020c020, 0xc420fee190) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc42020c020, 0xc420fee190) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 176 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200cc2c0, 0xc420552d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200cc2c0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a0018, 0xc4200b61e0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a0018, 0xc4200b61e0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 305 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4204ea000, 0xc420a44480, 0xc422f6bf20) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204ea000, 0xc421aa0f40) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 207 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd060, 0x72, 0xc4210fbce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420e8a618, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420e8a618, 0xc420f11000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420e8a600, 0xc420f11000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420e8a600, 0xc420f11000, 0x1000, 0x1000, 0xc420589dd8, 0x1, 0xc420589e08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc4205a0348, 0xc420f11000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc422f2a5a0, 0xc422781300, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc422f2a5a0, 0xc422781300, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc422f2a5a0, 0xc422781300, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42053a420, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42053a420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 386 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42287a180, 0xc42286f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc420f45bc0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).ReadByte(0xc420f45bc0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422226ed0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421aa1120) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420f45c20, 0x16e4b60, 0xc421aa11a0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420f45c20, 0x16e4b60, 0xc421aa11a0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422c7c140, 0x16e4b60, 0xc421aa11a0, 0xc421aa11a0, 0xc4200b77e8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422c7c140, 0xc421aa11a0, 0x2d, 0xc422213840) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequestHeader(0xc4200b77c0, 0x1d72340, 0xc422c7c140, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200b7cc0, 0x8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequest(0xc4200b77c0, 0x1d72340, 0xc422c7c140, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc4200b77c0, 0x1d72340, 0xc422c7c140, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc42287a180, 0xc4200b77c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 214 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42053a210) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 215 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42053a210) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 216 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420ff5320, 0xc420dd5700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4207623c0, 0x1d718c0, 0xc420ff5320, 0x0, 0xc420ff5300) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc420dd5950, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420868060, 0xc420d2ad80, 0xc420405e90, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.call(0xc4205e4720, 0xc42020d578, 0x13, 0x1bda9a0, 0x4, 0xc420dded48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.Call(0xc4205e4720, 0xc42020d578, 0x13, 0xc420dded48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*service).call(0xc420a7c8c0, 0xc420fee230, 0xc420e68178, 0x0, 0xc4211af300, 0xc420e7a0a0, 0x19758e0, 0xc420d2ad80, 0x16, 0x169e020, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc420fee230, 0x1d72340, 0xc420a7c900, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc4201760c0, 0xc420fee230) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 250 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd200, 0x72, 0xc4210f4ce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420c70018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420c70018, 0xc420524000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420c70000, 0xc420524000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420c70000, 0xc420524000, 0x1000, 0x1000, 0xc4210f4dd8, 0x0, 0xc4210f4e08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc420218180, 0xc420524000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc420fcc960, 0xc420f00870, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc420fcc960, 0xc420f00870, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc420fcc960, 0xc420f00870, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4203629a0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4203629a0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 240 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420363ce0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 251 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4203629a0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 252 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4203629a0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 331 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd105f198, 0x72, 0xc422a17ce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc4227d0718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc4227d0718, 0xc422ac1000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc4227d0700, 0xc422ac1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc4227d0700, 0xc422ac1000, 0x1000, 0x1000, 0xc422a17dd8, 0x1, 0xc422a17e08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc4205a1908, 0xc422ac1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc420d531a0, 0xc4218fa050, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc420d531a0, 0xc4218fa050, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc420d531a0, 0xc4218fa050, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4219cf810, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4219cf810) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 218 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd130, 0x72, 0xc4210f5ce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420c52598, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420c52598, 0xc420d01000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420c52580, 0xc420d01000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420c52580, 0xc420d01000, 0x1000, 0x1000, 0xc4210f5dd8, 0x0, 0xc4210f5e08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc4205a0010, 0xc420d01000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc42111fec0, 0xc42121d6a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc42111fec0, 0xc42121d6a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc42111fec0, 0xc42121d6a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4213de000, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4213de000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 338 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421883980, 0xc42311b700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4221bb350, 0x1d718c0, 0xc421883980, 0x0, 0xc421883900) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc42311b950, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420f442a0, 0xc420f3e180, 0xc4221bb230, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.call(0xc4205e4720, 0xc4202187c8, 0x13, 0x1bda9a0, 0x4, 0xc42135bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.Call(0xc4205e4720, 0xc4202187c8, 0x13, 0xc42135bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*service).call(0xc420dcf600, 0xc42051d810, 0xc421d85178, 0x0, 0xc4208d5780, 0xc4224b7520, 0x19758e0, 0xc420f3e180, 0x16, 0x169e020, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc42051d810, 0x1d72340, 0xc422636d80, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc420177800, 0xc42051d810) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 311 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd6e0, 0x72, 0xc4212df658) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc42185e498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc42185e498, 0xc42200c000, 0x1, 0x1) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc42185e480, 0xc42200c071, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc42185e480, 0xc42200c071, 0x1, 0x1, 0xc4212df7a0, 0x0, 0xc4231250e0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc4205a1e60, 0xc42200c071, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*connReader).backgroundRead(0xc42200c060) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 220 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4213de000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 297 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42023dd90) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 276 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4213defd0, 0xc420de0d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4213defd0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc422a180e8, 0xc422470af0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc422a180e8, 0xc422470af0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 277 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd610, 0x72, 0xc42111dce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420c70b18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420c70b18, 0xc4231cb000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420c70b00, 0xc4231cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420c70b00, 0xc4231cb000, 0x1000, 0x1000, 0xc42111ddd8, 0x1, 0xc42111de08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc422a180e8, 0xc4231cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc421c831a0, 0xc422f3c7f0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc421c831a0, 0xc422f3c7f0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc421c831a0, 0xc422f3c7f0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4213defd0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4213defd0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 296 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42023dd90) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 279 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4213defd0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 278 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4213defd0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 238 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd10bd880, 0x72, 0xc42135cce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420bad998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420bad998, 0xc422d56000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420bad980, 0xc422d56000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420bad980, 0xc422d56000, 0x1000, 0x1000, 0xc42135cdd8, 0x0, 0xc42135ce08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc422a18690, 0xc422d56000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc423090ae0, 0xc422ffb100, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc423090ae0, 0xc422ffb100, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc423090ae0, 0xc422ffb100, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420363ce0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420363ce0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 283 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420363ce0, 0xc420555d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420363ce0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc422a18690, 0xc422ce9db0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc422a18690, 0xc422ce9db0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 285 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd105f0c8, 0x72, 0xc421117ce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420bada98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420bada98, 0xc422165000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420bada80, 0xc422165000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420bada80, 0xc422165000, 0x1000, 0x1000, 0xc4212dfdd8, 0x1, 0xc4212dfe08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc422a186a8, 0xc422165000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc422455920, 0xc422f3c8b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc422455920, 0xc422f3c8b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc422455920, 0xc422f3c8b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4213df1e0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4213df1e0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 286 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4213df1e0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 287 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4213df1e0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 288 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420177500, 0xc422388000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc422408120) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).ReadByte(0xc422408120, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422a22120, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4230e2c60) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422408180, 0x16e4b60, 0xc4230e2cc0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422408180, 0x16e4b60, 0xc4230e2cc0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4230faac0, 0x16e4b60, 0xc4230e2cc0, 0xc4230e2cc0, 0xc4226be1b8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4230faac0, 0xc4230e2cc0, 0x2d, 0xc421d6f300) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequestHeader(0xc4226be190, 0x1d72340, 0xc4230faac0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200b7cc0, 0x8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequest(0xc4226be190, 0x1d72340, 0xc4230faac0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc4226be190, 0x1d72340, 0xc4230faac0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc420177500, 0xc4226be190) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 339 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420c9f8c0, 0xc42311f700, 0x20, 0x20, 0x169d4e0, 0xc4205f6e20) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4224e5650, 0x1d718c0, 0xc420c9f8c0, 0x0, 0xc420c9f800) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc42311f950, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc422fad080, 0xc421467500, 0xc4224e54d0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.call(0xc4205e4720, 0xc422a183f0, 0x13, 0x1bda9a0, 0x4, 0xc420dded48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.Call(0xc4205e4720, 0xc422a183f0, 0x13, 0xc420dded48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*service).call(0xc421fb90c0, 0xc422470b90, 0xc421d85240, 0x0, 0xc420bac300, 0xc4224b76a0, 0x19758e0, 0xc421467500, 0x16, 0x169e020, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc422470b90, 0x1d72340, 0xc422636ec0, 0x0, 0xc4201778c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc4201778c0, 0xc422470b90) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 208 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42053a420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 209 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42053a420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 306 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420a440c0, 0xc421b9a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc421f6ea20) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).ReadByte(0xc421f6ea20, 0x101000000000016, 0x0, 0x2d) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422864030, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422b02260) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421f6ea80, 0x16e4b60, 0xc422b022c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421f6ea80, 0x16e4b60, 0xc422b022c0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4230ce780, 0x16e4b60, 0xc422b022c0, 0xc422b022c0, 0xc422fa2348) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4230ce780, 0xc422b022c0, 0x2d, 0xc4227e2300) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequestHeader(0xc422fa2320, 0x1d72340, 0xc4230ce780, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200b7cc0, 0x8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequest(0xc422fa2320, 0x1d72340, 0xc4230ce780, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc422fa2320, 0x1d72340, 0xc4230ce780, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc420a440c0, 0xc422fa2320) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 330 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4219cf810, 0xc4210f7d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4219cf810, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a1908, 0xc4210a6910) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc4205a1908, 0xc4210a6910) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 344 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4225f4fd0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 295 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd105eff8, 0x72, 0xc4205ebce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc4226ee198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc4226ee198, 0xc420a0b000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc4226ee180, 0xc420a0b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc4226ee180, 0xc420a0b000, 0x1000, 0x1000, 0xc4212dfdd8, 0x1, 0xc4212dfe08) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc42000f2d0, 0xc420a0b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc421795800, 0xc4227c0db0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc421795800, 0xc4227c0db0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc421795800, 0xc4227c0db0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42023dd90, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42023dd90) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 332 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4219cf810) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 271 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421a63e60, 0xc42119f700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422799110, 0x1d718c0, 0xc421a63e60, 0x0, 0xc421a63e00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc42119f950, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420d0fbc0, 0xc422d2ce00, 0xc422798f90, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.call(0xc4205e4720, 0xc42000eeb8, 0x13, 0x1bda9a0, 0x4, 0xc42119fd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.Call(0xc4205e4720, 0xc42000eeb8, 0x13, 0xc42119fd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*service).call(0xc4212993c0, 0xc4212a0e10, 0xc4227c1c98, 0x0, 0xc420d27e00, 0xc4214a2060, 0x19758e0, 0xc422d2ce00, 0x16, 0x169e020, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc4212a0e10, 0x1d72340, 0xc422cfb800, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc420177680, 0xc4212a0e10) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 309 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd105ef28, 0x72, 0xc422a12ac8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc422d2c298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc422d2c298, 0xc421fbe000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc422d2c280, 0xc421fbe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc422d2c280, 0xc421fbe000, 0x1000, 0x1000, 0xc422ccc260, 0xc422a12be0, 0x7b9b0e) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc422a18de8, 0xc421fbe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc421ab7740) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).ReadByte(0xc421ab7740, 0x3, 0xc420040070, 0xc420040000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleCommand(0xc4203628f0, 0xc421ab7740, 0xc421ab77a0, 0xc420369dc0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:515 +0x43 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleConn(0xc4203628f0, 0x1d71840, 0xc4205fa400, 0x1d7d320, 0xc422a18de8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:499 +0x23d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:477 +0x1ce Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 342 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4225f4fd0, 0xc42135bd10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4225f4fd0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc422a19650, 0xc4200b7270) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc422a19650, 0xc4200b7270) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 333 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4219cf810) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 334 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420feca20, 0xc423121700, 0x20, 0x20, 0x169d4e0, 0xc4205f6e20) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422464900, 0x1d718c0, 0xc420feca20, 0x0, 0xc420feca00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc423121950, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420b07740, 0xc42185e080, 0xc4224647b0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.call(0xc4205e4720, 0xc4205a1c10, 0x13, 0x1bda9a0, 0x4, 0xc4210f8d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.Call(0xc4205e4720, 0xc4205a1c10, 0x13, 0xc4210f8d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*service).call(0xc42255d680, 0xc4210a69b0, 0xc4221a5f40, 0x0, 0xc420f3fa00, 0xc421548ce0, 0x19758e0, 0xc42185e080, 0x16, 0x169e020, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc4210a69b0, 0x1d72340, 0xc42255d6c0, 0x0, 0xc420085e00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc420085e00, 0xc4210a69b0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 335 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420188720, 0xc422ff92c8, 0x20, 0x20, 0x5, 0x1bd9a98) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42200c8a0, 0x1d718c0, 0xc420188720, 0x0, 0xc420188700) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc422ff9500, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Node).List(0xc4205e44e0, 0xc420188660, 0xc42200c450, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:1198 +0x27c Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.call(0xc4205e47e0, 0xc42020cd88, 0x13, 0x1bda9a0, 0x4, 0xc422ff98f8, 0x3, 0x3, 0x192ba60, 0x19, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.Call(0xc4205e47e0, 0xc42020cd88, 0x13, 0xc4230db8f8, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*service).call(0xc420213700, 0xc4202173b0, 0xc42146e2a8, 0x0, 0xc4205d7280, 0xc420b9c0e0, 0x1975760, 0xc420188660, 0x16, 0x169e120, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc4202173b0, 0x1d71b80, 0xc4229e61c0, 0xc420090000, 0xa) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc4200ee280, 0x1be205d, 0x9, 0x1975760, 0xc420188420, 0x169e120, 0xc42200c3c0, 0xc420188401, 0xc42200c3c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc4202ff2d0, 0x1be205d, 0x9, 0x1975760, 0xc420188420, 0x169e120, 0xc42200c3c0, 0xc42022c070, 0x1e) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest(0xc421298300, 0x1d6df80, 0xc420188180, 0xc420ecc600, 0xc42022c0a0, 0xc4230dbb58, 0x38, 0xc4230dbb50) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/node_endpoint.go:24 +0x1b3 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest-fm(0x1d6df80, 0xc420188180, 0xc420ecc600, 0xc420ecc600, 0xc4222120a0, 0x1e, 0xbed7c7fef58d41df) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:145 +0x48 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc420188180, 0xc420ecc600) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.HandlerFunc.ServeHTTP(0xc420ff9900, 0x1d6df80, 0xc420188180, 0xc420ecc600) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*ServeMux).ServeHTTP(0xc420ff7f80, 0x1d6df80, 0xc420188180, 0xc420ecc600) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc421cd8000, 0xc420ecc600) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.HandlerFunc.ServeHTTP(0xc420bb6270, 0x1d6fec0, 0xc421cd8000, 0xc420ecc600) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.serverHandler.ServeHTTP(0xc420640dd0, 0x1d6fec0, 0xc421cd8000, 0xc420ecc600) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*conn).serve(0xc422ff05a0, 0x1d71840, 0xc4229e6000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by net/http.(*Server).Serve Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 354 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420085ec0, 0xc42142e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc420fcdd40) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).ReadByte(0xc420fcdd40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422402d80, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4221741a0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420fcdda0, 0x16e4b60, 0xc422174200) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420fcdda0, 0x16e4b60, 0xc422174200, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422d87f80, 0x16e4b60, 0xc422174200, 0xc422174200, 0xc42051d838) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422d87f80, 0xc422174200, 0x2d, 0xc4221cd5e0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequestHeader(0xc42051d810, 0x1d72340, 0xc422d87f80, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200b7cc0, 0x8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).readRequest(0xc42051d810, 0x1d72340, 0xc422d87f80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc42051d810, 0x1d72340, 0xc422d87f80, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d0e0, 0xc420085ec0, 0xc42051d810) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 314 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd105ecb8, 0x72, 0xc422a16ce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420e32698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420e32698, 0xc420998000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420e32680, 0xc420998000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420e32680, 0xc420998000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc42000fc18, 0xc420998000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc4200877a0, 0xc4218e5110, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc4200877a0, 0xc4218e5110, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc4200877a0, 0xc4218e5110, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4209cb760, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4209cb760) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 312 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420f44420, 0xc42311d208, 0x20, 0x20, 0xc4227bac00, 0xc42200cf00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42200d290, 0x1d718c0, 0xc420f44420, 0x0, 0xc420f44400) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200ee280, 0xc42311d440, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Job).List(0xc42020c218, 0xc420f44300, 0xc42200cf00, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/job_endpoint.go:925 +0x2ea Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.call(0xc4201895c0, 0xc42020c700, 0x13, 0x1bda9a0, 0x4, 0xc42311d898, 0x3, 0x3, 0x192b7e0, 0x19, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: reflect.Value.Call(0xc4201895c0, 0xc42020c700, 0x13, 0xc4230db898, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*service).call(0xc420213400, 0xc4202173b0, 0xc42146e7e0, 0x0, 0xc42021db00, 0xc420b9c160, 0x1974e60, 0xc420f44300, 0x16, 0x169dd20, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/rpc.(*Server).ServeRequest(0xc4202173b0, 0x1d71b80, 0xc4229e68c0, 0xc420057c00, 0xa) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc4200ee280, 0x1be08be, 0x8, 0x1974e60, 0xc420f44120, 0x169dd20, 0xc42200ce70, 0xc420f44101, 0xc42200ce70) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc4202ff2d0, 0x1be08be, 0x8, 0x1974e60, 0xc420f44120, 0x169dd20, 0xc42200ce70, 0x0, 0xc420057c00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).jobListRequest(0xc421298300, 0x1d6df80, 0xc420189140, 0xc420ecc800, 0x44774f, 0xc42129be00, 0xc42022c310, 0x1d) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:33 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest(0xc421298300, 0x1d6df80, 0xc420189140, 0xc420ecc800, 0xc42022c340, 0xc4230dbb58, 0x38, 0xc4230dbb50) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:18 +0x1a6 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest-fm(0x1d6df80, 0xc420189140, 0xc420ecc800, 0xc420ecc800, 0xc422212320, 0x1d, 0xbed7c7fef5998631) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:141 +0x48 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc420189140, 0xc420ecc800) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.HandlerFunc.ServeHTTP(0xc420ff98a0, 0x1d6df80, 0xc420189140, 0xc420ecc800) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*ServeMux).ServeHTTP(0xc420ff7f80, 0x1d6df80, 0xc420189140, 0xc420ecc800) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc421cd80e0, 0xc420ecc800) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.HandlerFunc.ServeHTTP(0xc420bb6270, 0x1d6fec0, 0xc421cd80e0, 0xc420ecc800) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.serverHandler.ServeHTTP(0xc420640dd0, 0x1d6fec0, 0xc421cd80e0, 0xc420ecc800) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*conn).serve(0xc4217480a0, 0x1d71840, 0xc4229e6580) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by net/http.(*Server).Serve Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 313 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd105ed88, 0x72, 0xc421710658) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc420e32298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc420e32298, 0xc42200cb00, 0x1, 0x1) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc420e32280, 0xc42200cbb1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc420e32280, 0xc42200cbb1, 0x1, 0x1, 0xc420376420, 0xf9896a, 0x1) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc42000fc00, 0xc42200cbb1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net/http.(*connReader).backgroundRead(0xc42200cba0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 315 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4209cb760) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 316 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4209cb760) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 348 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd105eb18, 0x72, 0xc4210f6858) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc42185e818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc42185e818, 0xc420e74000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc42185e800, 0xc420e74000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc42185e800, 0xc420e74000, 0x1000, 0x1000, 0xc420046f00, 0xc422ccc618, 0xc422ccc620) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc422a199a0, 0xc420e74000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).fill(0xc42111e420) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).ReadByte(0xc42111e420, 0x72, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421f3db30, 0xc42185e818) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4224b7bc0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42111e480, 0x16e76e0, 0xc421874370) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42111e480, 0x16e76e0, 0xc421874370, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.decodeResponse(0xc422ca60c0, 0x182ba60, 0xc4224563e0, 0xc420fda960, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:608 +0x5e Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).genericRPC(0xc4203628f0, 0xc420214690, 0x24, 0xc42034a6a0, 0x11, 0x32b98bb20fda900, 0x182ba00, 0xc420fda960, 0x182ba60, 0xc4224563e0, ...) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:406 +0xf9 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).AppendEntries(0xc4203628f0, 0xc420214690, 0x24, 0xc42034a6a0, 0x11, 0xc420fda960, 0xc4224563e0, 0x1, 0x3) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:379 +0x97 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4204ea000, 0xc420a443c0, 0x18634b, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:186 +0x189 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4204ea000, 0xc420a443c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:132 +0x3f2 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204ea000, 0xc4224b7ce0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 357 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc421eb09c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 320 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4204ea000, 0xc420a443c0, 0xc423125c80) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204ea000, 0xc42213c2c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 350 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4219cfad0, 0xc4230d9d10, 0x30, 0xf9866a) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4219cfad0, 0x1c61988, 0xc4200ee280, 0x1d71840, 0xc421298240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc422a19a20, 0xc4200b7720) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200ee280, 0x1d71840, 0xc421298240, 0x1d7d320, 0xc422a19a20, 0xc4200b7720) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 351 [IO wait]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.runtime_pollWait(0x7f7dd105ea48, 0x72, 0xc422e91ce8) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).wait(0xc42185eb98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*pollDesc).waitRead(0xc42185eb98, 0xc421c53000, 0x1000, 0x1000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: internal/poll.(*FD).Read(0xc42185eb80, 0xc421c53000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*netFD).Read(0xc42185eb80, 0xc421c53000, 0x1000, 0x1000, 0x8e, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: net.(*conn).Read(0xc422a19a20, 0xc421c53000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: bufio.(*Reader).Read(0xc42111efc0, 0xc4218fa710, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadAtLeast(0x1d59aa0, 0xc42111efc0, 0xc4218fa710, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: io.ReadFull(0x1d59aa0, 0xc42111efc0, 0xc4218fa710, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4219cfad0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4219cfad0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 352 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4219cfad0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 353 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4219cfad0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 372 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4207ab900) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 358 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4204ea000, 0xc420a44480, 0x1d6eb80, 0xc421eb09c0, 0xc42309d500, 0xc42309d560) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4204ea000, 0xc421eb0a00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 373 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4207ab900) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 374 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc420217360, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4200ee280) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 375 [semacquire]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: sync.runtime_SemacquireMutex(0xc4200d412c, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: sync.(*Mutex).Lock(0xc4200d4128) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: sync.(*RWMutex).Lock(0xc4200d4128) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4200d40a0, 0x1d71840, 0xc422ca7b40) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 376 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc42053e120) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 377 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc42053e120) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 378 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc4221ae240) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 379 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422ca7b80, 0xc420ddece8, 0x20, 0x20, 0x18, 0xc4221747c0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42219c2a0, 0x1d71840, 0xc422ca7b80, 0xc4221747c0, 0x184987) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42059bca0, 0xc420dd9ed0, 0x184987, 0x1d71840, 0xc422ca7b80, 0xc420e8a360, 0xc4221ae180, 0xc420b1c300, 0x24, 0x2bba400) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420e8a300, 0x1d71840, 0xc422ca7b80, 0x184987, 0xc421eb0b00, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420e8a300, 0x1d71840, 0xc422ca7b80) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 380 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422ca7c00, 0xc422ff5770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42219c7e0, 0x1d71840, 0xc422ca7c00, 0x0, 0x1849b5) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42059bca0, 0xc422ff5960, 0x1849b5, 0x1d71840, 0xc422ca7c00, 0x552bde, 0xc422ff5950, 0xc422ff5970, 0xf33ba5, 0xc42014d868) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc42014d810, 0x1d71840, 0xc422ca7c00, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc42014d810) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 381 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422ca7bc0, 0xc422fb7a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4222274d0, 0x1d71840, 0xc422ca7bc0, 0xc422227680, 0x1833a4) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42059bca0, 0xc422fb7c68, 0x1833a4, 0x1d71840, 0xc422ca7bc0, 0x552bde, 0xc4202170e0, 0x2, 0xc4207196c0, 0x40) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422ca7c40, 0x1833a4, 0x3b, 0xc422e8dde8, 0x1, 0x1) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422ca7c40) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 382 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422ca7c80) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 383 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420cf85a0, 0x1d71840, 0xc422ca7bc0) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 387 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422227260, 0xc4230fc300) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 388 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422227260, 0xc4230fc360) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 359 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc422ff0f00) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: goroutine 321 [select]: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc421eb0bc0, 0xc4228e1d28, 0x20, 0x20, 0xc420057800, 0x1bd9a98) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421f959e0, 0x1d71840, 0xc421eb0bc0, 0xc4221749e0, 0x1849b5) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42059bca0, 0xc42267bf20, 0x1849b5, 0x1d71840, 0xc421eb0bc0, 0xc422e87940, 0x1a, 0xc421eb1700, 0x1b, 0xc42070a180) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc422ff0f00, 0x1849b5, 0x22, 0xc420da5f00, 0x23, 0xc4226fca00, 0x24, 0xc4225c4000) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc422ff0f00, 0x1849b5, 0xc420fdb080) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25889]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:28:45 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:28:45 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:28:45 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: ==> Starting Nomad agent... Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: ==> Nomad agent configuration: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: Client: false Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: Log Level: INFO Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: Region: amrv01 (DC: test) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: Server: true Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: Version: 0.8.4 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:45 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:45 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:45 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:45 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:45.721966 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:45 [INFO] serf: Attempting re-join to previously known node: ip-172-25-5-210.amv01: 172.25.5.210:4648 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:45.722293 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:45 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:28:45 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:45.746801 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.076926 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.076951 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.076962 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.076974 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.076985 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.076997 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [WARN] memberlist: Refuting an alive message Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46 [INFO] serf: Re-joined to previously known node: ip-172-25-5-210.amv01: 172.25.5.210:4648 Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.077936 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.077949 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.077960 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.077970 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.077981 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.077992 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.078003 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.078014 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.078024 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.078036 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.078048 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.078058 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:28:46 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:46.078072 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:28:47 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:47 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:28:47 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:47 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 55 Aug 23 15:28:47 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:47 [INFO] raft: Election won. Tally: 2 Aug 23 15:28:47 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:47 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:28:47 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:47 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:28:47 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:47 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:28:47 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:47.722059 [INFO] nomad: cluster leadership acquired Aug 23 15:28:47 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:47 [WARN] raft: AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} rejected, sending older logs (next: 1598284) Aug 23 15:28:47 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:47 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:28:47 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:47 [INFO] raft: pipelining replication to peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:48 ip-172-24-20-236 nomad.sh[25967]: 2018/08/23 15:28:48 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: fatal error: concurrent map read and map write Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 222 [running]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc420f7ad20 sp=0xc420f7ad00 pc=0x42d161 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: runtime.mapaccess2(0x1844be0, 0xc42002f1a0, 0xc420f7ade0, 0xc42000e048, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc420f7ad68 sp=0xc420f7ad20 pc=0x409c75 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc420380000, 0x1d6dd40, 0xc42000e048, 0x0, 0xed3109f81) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:156 +0xdc fp=0xc420f7ae50 sp=0xc420f7ad68 pc=0xf215ac Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc420338a00, 0x1d71840, 0xc422b3df00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc420f7afc8 sp=0xc420f7ae50 pc=0xf56bb7 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: runtime.goexit() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420f7afd0 sp=0xc420f7afc8 pc=0x45c711 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 1 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420131730, 0xc4202b1400) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420131730, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4203e9400, 0xc4203e9400, 0xc420213500, 0x37) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204d7f68) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: main.main() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 5 [syscall]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: os/signal.signal_recv(0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: os/signal.loop() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by os/signal.init.0 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 25 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4201ffa00, 0x1d71840, 0xc42042c180) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 120 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200b4dc0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4dc0, 0xc42068a690) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 103 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205ae160, 0xc4214a9d10, 0x30, 0xf9866a) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205ae160, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4210fa1c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc4202107a0, 0xc4210f4550) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc4202107a0, 0xc4210f4550) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 39 [select, locked to thread]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: runtime.selectgo(0xc420050750, 0xc420090720) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: runtime.ensureSigM.func1() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: runtime.goexit() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 40 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc4201ff940) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 41 [sleep]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: time.Sleep(0x3b9aca00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420439ad0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 42 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420472540) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 121 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200b4dc0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4dc0, 0xc42068a6a0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 43 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4201f6ea0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 45 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc42036c750) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc42002f5f0, 0xc4203ebe60) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 78 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420f64240, 0xc420718000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc4211e77a0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc4211e77a0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4214c3680, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42135b2c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4211e7800, 0x16e4b60, 0xc42135b320) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4211e7800, 0x16e4b60, 0xc42135b320, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420a26800, 0x16e4b60, 0xc42135b320, 0xc42135b320, 0xc4210f47f8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420a26800, 0xc42135b320, 0x2d, 0xc422cf8a00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc4210f47d0, 0x1d72340, 0xc420a26800, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc4210f47d0, 0x1d72340, 0xc420a26800, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc4210f47d0, 0x1d72340, 0xc420a26800, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc420f64240, 0xc4210f47d0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 53 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4200a9630, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4205ae000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 72 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cde30, 0x72, 0xc4205239a8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420376198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420376198, 0xc4205ce000, 0x1000, 0x1000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Read(0xc420376180, 0xc4205ce000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).Read(0xc420376180, 0xc4205ce000, 0x1000, 0x1000, 0x458600, 0xc42028f800, 0x4) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*conn).Read(0xc420204208, 0xc4205ce000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/http.(*persistConn).Read(0xc420392120, 0xc4205ce000, 0x1000, 0x1000, 0xc420523b98, 0x405675, 0xc420482600) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc42031ab40) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).Peek(0xc42031ab40, 0x1, 0x0, 0x0, 0x1, 0xc420db98c0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/http.(*persistConn).readLoop(0xc420392120) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by net/http.(*Transport).dialConn Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 73 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/http.(*persistConn).writeLoop(0xc420392120) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by net/http.(*Transport).dialConn Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 119 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200b4dc0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200b4dc0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200b4dc0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4dc0, 0xc42068a680) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 122 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc42110cf60) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 123 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420f0a680) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 77 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42059a840, 0xc420d7c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc4211e74a0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc4211e74a0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4214c3260, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42135b0e0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4211e7500, 0x16e4b60, 0xc42135b140) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4211e7500, 0x16e4b60, 0xc42135b140, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420a26680, 0x16e4b60, 0xc42135b140, 0xc42135b140, 0xc4210f47f8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420a26680, 0xc42135b140, 0x2d, 0xc420b7eec0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc4210f47d0, 0x1d72340, 0xc420a26680, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc4210f47d0, 0x1d72340, 0xc420a26680, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc4210f47d0, 0x1d72340, 0xc420a26680, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc42059a840, 0xc4210f47d0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 104 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205ae0b0, 0xc4214a7d10, 0x30, 0xf9866a) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205ae0b0, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4210fa1c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc4202107a8, 0xc4210f45f0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc4202107a8, 0xc4210f45f0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 124 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cdd60, 0x72, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420218098, 0x72, 0xc420a26100, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420218098, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Accept(0xc420218080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).accept(0xc420218080, 0xc420204380, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*TCPListener).accept(0xc4204a2228, 0x458e40, 0xc4204b3f90, 0xc4204b3f98) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*TCPListener).AcceptTCP(0xc4204a2228, 0x1c63658, 0xc420ca03c0, 0xc420204380) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:28:49 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420ca03c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 125 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cdc90, 0x72, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420218218, 0x72, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420218218, 0xc4221fc000, 0x10000, 0x10000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).ReadFrom(0xc420218200, 0xc4221fc000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).readFrom(0xc420218200, 0xc4221fc000, 0x10000, 0x10000, 0x340, 0x340, 0x1d62b40, 0x412539, 0xc42051fe40) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*UDPConn).readFrom(0xc4204a2280, 0xc4221fc000, 0x10000, 0x10000, 0x101, 0xc42051fe18, 0xc42051fe90, 0x443cf7) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*UDPConn).ReadFrom(0xc4204a2280, 0xc4221fc000, 0x10000, 0x10000, 0x10000, 0x10000, 0xf4d8, 0xb3af955b, 0x2b96de0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420ca03c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 126 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420ca03c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 127 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420ca03c0, 0x12a05f200, 0xc421122720, 0xc420f10cc0, 0xc42068a5f0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 128 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420ca03c0, 0xc420f10cc0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 129 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420ca03c0, 0x1dcd6500, 0xc421122780, 0xc420f10cc0, 0xc42068a600) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 146 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc420574c00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 147 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc420574c00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 148 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420574c00, 0x1bdd09a, 0x6, 0xc42110d0b0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 149 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420574c00, 0x1bdba6c, 0x5, 0xc42110d0e0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 150 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420574c00, 0x1bdbc6a, 0x5, 0xc42110d110) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 81 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420f649c0, 0xc420abf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc420958f60) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc420958f60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a8b920, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420c0f700) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420958fc0, 0x16e4b60, 0xc420c0f760) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420958fc0, 0x16e4b60, 0xc420c0f760, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420a27a80, 0x16e4b60, 0xc420c0f760, 0xc420c0f760, 0xc4210f47f8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420a27a80, 0xc420c0f760, 0x2d, 0xc42232c1c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc4210f47d0, 0x1d72340, 0xc420a27a80, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc4210f47d0, 0x1d72340, 0xc420a27a80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc4210f47d0, 0x1d72340, 0xc420a27a80, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc420f649c0, 0xc4210f47d0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 152 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4211227e0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4211227e0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4211227e0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 153 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421122840, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421122840, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421122840) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 154 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4211228a0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4211228a0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4211228a0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 155 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421122900, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421122900, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421122900) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 156 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cdaf0, 0x77, 0xc42028f800) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc422c14118, 0x77, 0xc42103d000, 0xc42103d028, 0x436544) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitWrite(0xc422c14118, 0xc42103d000, 0x28, 0x611b26) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).WaitWrite(0xc422c14100, 0x1c65988, 0x1d718c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:440 +0x37 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).connect(0xc422c14100, 0x1d718c0, 0xc4227c2780, 0x0, 0x0, 0x1d5c800, 0xc421c433e0, 0x0, 0x0, 0x0, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:152 +0x299 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).dial(0xc422c14100, 0x1d718c0, 0xc4227c2780, 0x1d7a200, 0x0, 0x1d7a200, 0xc422ca40f0, 0xc42103d248, 0x62f2be) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/sock_posix.go:142 +0xe9 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.socket(0x1d718c0, 0xc4227c2780, 0x1bda392, 0x3, 0x2, 0x1, 0x0, 0x0, 0x1d7a200, 0x0, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/sock_posix.go:93 +0x1a6 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.internetSocket(0x1d718c0, 0xc4227c2780, 0x1bda392, 0x3, 0x1d7a200, 0x0, 0x1d7a200, 0xc422ca40f0, 0x1, 0x0, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/ipsock_posix.go:141 +0x129 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.doDialTCP(0x1d718c0, 0xc4227c2780, 0x1bda392, 0x3, 0x0, 0xc422ca40f0, 0x2bb8aa0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/tcpsock_posix.go:62 +0xb9 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.dialTCP(0x1d718c0, 0xc4227c2780, 0x1bda392, 0x3, 0x0, 0xc422ca40f0, 0x7c5043c0b, 0x2b96de0, 0x6fc23725d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/tcpsock_posix.go:58 +0xe4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.dialSingle(0x1d718c0, 0xc4227c2780, 0xc422c14080, 0x1d62b00, 0xc422ca40f0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/dial.go:547 +0x375 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.dialSerial(0x1d718c0, 0xc4227c2780, 0xc422c14080, 0xc422d423a0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/dial.go:515 +0x22d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*Dialer).DialContext(0xc42103d848, 0x1d71880, 0xc42003e028, 0x1bda392, 0x3, 0xc422e51df0, 0x10, 0x0, 0x0, 0x0, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/dial.go:397 +0x678 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*Dialer).Dial(0xc42103d848, 0x1bda392, 0x3, 0xc422e51df0, 0x10, 0x20, 0x20, 0x17503e0, 0x2ade3c8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/dial.go:320 +0x75 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc420ca03c0, 0xc422e51dc0, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:758 +0x14e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc420ca03c0, 0xc422e51dc0, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc420ca03c0, 0xc4201dca00, 0x14, 0x20, 0x3, 0x3, 0xc421111350) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc420574c00, 0xc4201dca00, 0x14, 0x20, 0xc421111301, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc4200e1900, 0xc4201dca00, 0x14, 0x20, 0xc421111350, 0x10, 0xc4206854c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc420352f40, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc42110def0, 0xc4200e1900) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 157 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4200e1900) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 158 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4200e1900) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 159 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cdf00, 0x72, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420399a98, 0x72, 0xc4201fea00, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420399a98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Accept(0xc420399a80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).accept(0xc420399a80, 0x1, 0x1, 0xc420566f20) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*TCPListener).accept(0xc420210790, 0x406b4a, 0xc420f97440, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*TCPListener).Accept(0xc420210790, 0xc420f97440, 0x0, 0x3f800000, 0xc4210fa1c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4200e1900, 0x1d71840, 0xc4210fa1c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 160 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc420338a00, 0x3b9aca00, 0xc420090ba0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 161 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc42020adc0, 0x3b9aca00, 0xc420090ba0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 162 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420450c60, 0x3b9aca00, 0xc420090ba0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 163 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc42036c750, 0x3b9aca00, 0xc420090ba0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 164 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4200e1900) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 165 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cdbc0, 0x72, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420c90718, 0x72, 0xc4208a6900, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420c90718, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Accept(0xc420c90700, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).accept(0xc420c90700, 0x412d88, 0x30, 0x1a1fac0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*TCPListener).accept(0xc4204a2ba0, 0x5fea4a, 0x1a1fac0, 0xc4214c2510) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*TCPListener).Accept(0xc4204a2ba0, 0xc42003e028, 0x1830320, 0x2ab0680, 0x1b7b4c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/http.(*Server).Serve(0xc420394410, 0x1d6fc00, 0xc4204a2ba0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/http.Serve(0x1d6fc00, 0xc4204a2ba0, 0x1d5df00, 0xc4214c2450, 0x4, 0xc420d43f00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc4210fa280, 0x1d6fc00, 0xc4204a2ba0, 0xc42068ab80, 0xc42110dfb0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 166 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420c90a00, 0xc42068ab90, 0xc420f11560) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 167 [sleep]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: time.Sleep(0xd5563bc2) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc420131730, 0xc420c90a00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 105 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cd950, 0x72, 0xc4214aace8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420c22698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420c22698, 0xc42110a000, 0x1000, 0x1000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Read(0xc420c22680, 0xc42110a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).Read(0xc420c22680, 0xc42110a000, 0x1000, 0x1000, 0xc4200555d8, 0x1, 0xc420055608) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*conn).Read(0xc4202107a8, 0xc42110a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).Read(0xc421380180, 0xc4211101b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadAtLeast(0x1d59aa0, 0xc421380180, 0xc4211101b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadFull(0x1d59aa0, 0xc421380180, 0xc4211101b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205ae0b0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205ae0b0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 106 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205ae0b0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 107 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205ae0b0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 108 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cda20, 0x72, 0xc4214a4ce8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420c22518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420c22518, 0xc42110b000, 0x1000, 0x1000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Read(0xc420c22500, 0xc42110b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).Read(0xc420c22500, 0xc42110b000, 0x1000, 0x1000, 0xc4214a4dd8, 0x0, 0xc4214a4e08) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*conn).Read(0xc4202107a0, 0xc42110b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).Read(0xc42112eb40, 0xc4211103c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadAtLeast(0x1d59aa0, 0xc42112eb40, 0xc4211103c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadFull(0x1d59aa0, 0xc42112eb40, 0xc4211103c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205ae160, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205ae160) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 109 [runnable]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: syscall.Syscall(0x1, 0xc, 0xc42110fb10, 0xc, 0xc, 0xc, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: syscall.write(0xc, 0xc42110fb10, 0xc, 0xc, 0xc4230de600, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/syscall/zsyscall_linux_amd64.go:1022 +0x5f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: syscall.Write(0xc, 0xc42110fb10, 0xc, 0xc, 0xc420f5ec60, 0xc42112ec00, 0xc420f5ec60) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/syscall/syscall_unix.go:181 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Write(0xc420c22500, 0xc42110fb10, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:256 +0x142 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).Write(0xc420c22500, 0xc42110fb10, 0xc, 0xc, 0xbba0ae, 0xc42044bf80, 0xc4214a5e38) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:220 +0x4f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*conn).Write(0xc4202107a0, 0xc42110fb10, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/net.go:188 +0x6a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205ae160) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:398 +0xaa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 110 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205ae160) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 111 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420140240, 0xc421046000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc420eeb3e0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc420eeb3e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420017f80, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4210fc480) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420eeb440, 0x16e4b60, 0xc4210fc4e0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420eeb440, 0x16e4b60, 0xc4210fc4e0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42112cd40, 0x16e4b60, 0xc4210fc4e0, 0xc4210fc4e0, 0xc4210f46b8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42112cd40, 0xc4210fc4e0, 0x2d, 0xc42065b5c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc4210f4690, 0x1d72340, 0xc42112cd40, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc4210f4690, 0x1d72340, 0xc42112cd40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc4210f4690, 0x1d72340, 0xc42112cd40, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc420140240, 0xc4210f4690) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 244 [runnable]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc421126410, 0x7f9ff6006390, 0xc422824180, 0x1, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:248 +0x2c1 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc421126410, 0x7f9ff6006390, 0xc422824180, 0x7f9ff6006390, 0xc422824180) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).latestEvalIndex(0xc4207d7400, 0xc420564ef8, 0xc420564ef8, 0x4) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:784 +0x89 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).handleAllocUpdate(0xc4207d7400, 0xc420210f10, 0x1, 0x1, 0xd0f6849d, 0x2b96de0, 0x3354fabebb, 0x0, 0xc42034e6d0, 0x43e2b2) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:514 +0x40 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc4207d7400) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:441 +0x591 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 220 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420f3f2c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 229 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422ac21e0, 0xc421eb6060) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 198 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420439760, 0xc420562d10, 0x30, 0xf9866a) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420439760, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4210fa1c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc420204b88, 0xc42043f310) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc420204b88, 0xc42043f310) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 142 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4203a0210, 0xc420eedd10, 0x30, 0xf9866a) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4203a0210, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4210fa1c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc4202042a8, 0xc420c2a500) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc4202042a8, 0xc420c2a500) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 187 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4203a1290, 0xc420ce3d10, 0x30, 0xf9866a) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4203a1290, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4210fa1c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc420211168, 0xc4207e2a00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc420211168, 0xc4207e2a00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 182 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cd880, 0x72, 0xc4200629a8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc421061d98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc421061d98, 0xc4214b2000, 0x1000, 0x1000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Read(0xc421061d80, 0xc4214b2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).Read(0xc421061d80, 0xc4214b2000, 0x1000, 0x1000, 0x458600, 0xc42048b380, 0x4) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*conn).Read(0xc420210f30, 0xc4214b2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/http.(*persistConn).Read(0xc4214b6000, 0xc4214b2000, 0x1000, 0x1000, 0xc420062b98, 0x405675, 0xc420ffdf20) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc4204ae8a0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).Peek(0xc4204ae8a0, 0x1, 0x0, 0x0, 0x1, 0xc420eccf00, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/http.(*persistConn).readLoop(0xc4214b6000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by net/http.(*Transport).dialConn Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 183 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/http.(*persistConn).writeLoop(0xc4214b6000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by net/http.(*Transport).dialConn Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 168 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42059a480, 0xc421107000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc421122b40) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc421122b40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4214b00c0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421125820) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421122ba0, 0x16e4b60, 0xc421125880) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421122ba0, 0x16e4b60, 0xc421125880, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4210fa300, 0x16e4b60, 0xc421125880, 0xc421125880, 0xc4210f47f8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4210fa300, 0xc421125880, 0x2d, 0xc42232c0c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc4210f47d0, 0x1d72340, 0xc4210fa300, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc4210f47d0, 0x1d72340, 0xc4210fa300, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc4210f47d0, 0x1d72340, 0xc4210fa300, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc42059a480, 0xc4210f47d0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 169 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42059a540, 0xc420c5d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc421122d80) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc421122d80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4214b0240, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421125980) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421122de0, 0x16e4b60, 0xc4211259e0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421122de0, 0x16e4b60, 0xc4211259e0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4210fa3c0, 0x16e4b60, 0xc4211259e0, 0xc4211259e0, 0xc4210f47f8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4210fa3c0, 0xc4211259e0, 0x2d, 0xc420340460) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc4210f47d0, 0x1d72340, 0xc4210fa3c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc4210f47d0, 0x1d72340, 0xc4210fa3c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc4210f47d0, 0x1d72340, 0xc4210fa3c0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc42059a540, 0xc4210f47d0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 170 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cd7b0, 0x72, 0xc420ef2ce8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420ba2318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420ba2318, 0xc420319000, 0x1000, 0x1000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Read(0xc420ba2300, 0xc420319000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).Read(0xc420ba2300, 0xc420319000, 0x1000, 0x1000, 0xc420ef2dd8, 0x201, 0xc420ef2e08) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*conn).Read(0xc4202042a8, 0xc420319000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).Read(0xc421122f60, 0xc421110eb0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadAtLeast(0x1d59aa0, 0xc421122f60, 0xc421110eb0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadFull(0x1d59aa0, 0xc421122f60, 0xc421110eb0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4203a0210, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4203a0210) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 171 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4203a0210) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 172 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4203a0210) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 230 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422ac21e0, 0xc421eb60c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 174 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42059a600, 0xc420ffe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc42044bce0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc42044bce0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4214b19b0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421125c00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42044bd40, 0x16e4b60, 0xc421125c60) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42044bd40, 0x16e4b60, 0xc421125c60, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4210fa980, 0x16e4b60, 0xc421125c60, 0xc421125c60, 0xc421126668) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4210fa980, 0xc421125c60, 0x2d, 0xc422cf8980) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc421126640, 0x1d72340, 0xc4210fa980, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc421126640, 0x1d72340, 0xc4210fa980, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc421126640, 0x1d72340, 0xc4210fa980, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc42059a600, 0xc421126640) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 175 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).waitForSendErr(0xc4205ae160, 0xc42110fb10, 0xc, 0xc, 0x0, 0x0, 0xc42044bf20, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:353 +0x3ca Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).sendClose(0xc42059a6c0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:281 +0xfd Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Close(0xc42059a6c0, 0x0, 0xc42059a600) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:316 +0x7d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc42059a6c0, 0xc4210f47d0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:241 +0x2f6 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 176 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42059a780, 0xc420f60000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc420536a20) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc420536a20, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4214b1cb0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421125f20) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420536a80, 0x16e4b60, 0xc421125f80) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420536a80, 0x16e4b60, 0xc421125f80, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4210fab00, 0x16e4b60, 0xc421125f80, 0xc421125f80, 0xc421126668) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4210fab00, 0xc421125f80, 0x2d, 0xc42232c000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc421126640, 0x1d72340, 0xc4210fab00, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc421126640, 0x1d72340, 0xc4210fab00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc421126640, 0x1d72340, 0xc4210fab00, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc42059a780, 0xc421126640) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 177 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42059a900, 0xc4206cc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc420536f00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc420536f00, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42057a930, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4213161c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420536f60, 0x16e4b60, 0xc421316220) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420536f60, 0x16e4b60, 0xc421316220, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4210fac40, 0x16e4b60, 0xc421316220, 0xc421316220, 0xc4210f47f8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4210fac40, 0xc421316220, 0x2d, 0xc422158520) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc4210f47d0, 0x1d72340, 0xc4210fac40, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc4210f47d0, 0x1d72340, 0xc4210fac40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc4210f47d0, 0x1d72340, 0xc4210fac40, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc42059a900, 0xc4210f47d0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 192 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420472d80, 0xc420367000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc420d9a300) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc420d9a300, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420cf9950, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420be3940) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d9a360, 0x16e4b60, 0xc420be39a0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d9a360, 0x16e4b60, 0xc420be39a0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420ab2c80, 0x16e4b60, 0xc420be39a0, 0xc420be39a0, 0xc4210f47f8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420ab2c80, 0xc420be39a0, 0x2d, 0xc4221584c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc4210f47d0, 0x1d72340, 0xc420ab2c80, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc4210f47d0, 0x1d72340, 0xc420ab2c80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc4210f47d0, 0x1d72340, 0xc420ab2c80, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc420472d80, 0xc4210f47d0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 199 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cd6e0, 0x72, 0xc420ce5ce8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420ecf998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420ecf998, 0xc4209bb000, 0x1000, 0x1000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Read(0xc420ecf980, 0xc4209bb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).Read(0xc420ecf980, 0xc4209bb000, 0x1000, 0x1000, 0xc420042501, 0x212, 0x1eb10e0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*conn).Read(0xc420204b88, 0xc4209bb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).Read(0xc420959d40, 0xc42079c700, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadAtLeast(0x1d59aa0, 0xc420959d40, 0xc42079c700, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadFull(0x1d59aa0, 0xc420959d40, 0xc42079c700, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420439760, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420439760) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 200 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420439760) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 201 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420439760) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 202 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421123e00, 0xc42130f700, 0x20, 0x20, 0x169d4e0, 0xc42072f3c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42190c150, 0x1d718c0, 0xc421123e00, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1900, 0xc42130f950, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420e0db00, 0xc4213ba000, 0xc420f20c30, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: reflect.Value.call(0xc4200872c0, 0xc420204e90, 0x13, 0x1bda9a0, 0x4, 0xc42130fd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: reflect.Value.Call(0xc4200872c0, 0xc420204e90, 0x13, 0xc42130fd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*service).call(0xc420c38200, 0xc42043f3b0, 0xc42077e8a8, 0x0, 0xc420e1e280, 0xc420c0ff20, 0x19758e0, 0xc4213ba000, 0x16, 0x169e020, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc42043f3b0, 0x1d72340, 0xc420c38a40, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc420140780, 0xc42043f3b0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 203 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42059ae40, 0xc420c6a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc420e30960) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc420e30960, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4208c76e0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42071c280) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420e309c0, 0x16e4b60, 0xc42071c2e0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420e309c0, 0x16e4b60, 0xc42071c2e0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420b31880, 0x16e4b60, 0xc42071c2e0, 0xc42071c2e0, 0xc4210f47f8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420b31880, 0xc42071c2e0, 0x2d, 0xc420b7e940) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc4210f47d0, 0x1d72340, 0xc420b31880, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc4210f47d0, 0x1d72340, 0xc420b31880, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc4210f47d0, 0x1d72340, 0xc420b31880, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc42059ae40, 0xc4210f47d0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 188 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cd610, 0x72, 0xc42149dce8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420ba2998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420ba2998, 0xc420c6c000, 0x1000, 0x1000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Read(0xc420ba2980, 0xc420c6c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).Read(0xc420ba2980, 0xc420c6c000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*conn).Read(0xc420211168, 0xc420c6c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).Read(0xc420bc5620, 0xc4207da2c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadAtLeast(0x1d59aa0, 0xc420bc5620, 0xc4207da2c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadFull(0x1d59aa0, 0xc420bc5620, 0xc4207da2c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4203a1290, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4203a1290) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 189 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4203a1290) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 190 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4203a1290) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 191 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420dd57a0, 0xc42111d700, 0x20, 0x20, 0x169d4e0, 0xc42037dd60) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42094eae0, 0x1d718c0, 0xc420dd57a0, 0x0, 0xc420dd5700) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1900, 0xc42111d950, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420f6b3e0, 0xc420ece380, 0xc42094e960, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: reflect.Value.call(0xc4200872c0, 0xc420211470, 0x13, 0x1bda9a0, 0x4, 0xc42111dd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: reflect.Value.Call(0xc4200872c0, 0xc420211470, 0x13, 0xc42111dd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*service).call(0xc420f4bf80, 0xc4207e2aa0, 0xc42057e128, 0x0, 0xc420c65280, 0xc420be3480, 0x19758e0, 0xc420ece380, 0x16, 0x169e020, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc4207e2aa0, 0x1d72340, 0xc420f4bfc0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc420f64a80, 0xc4207e2aa0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 215 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b4dc0, 0xc420f64d80, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b4dc0, 0xc420f64d80) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4dc0, 0xc420af2d00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 216 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b4dc0, 0xc420f64e40, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b4dc0, 0xc420f64e40) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4dc0, 0xc420af2d20) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 218 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b4dc0, 0xc420f64d80, 0x1d6eb80, 0xc4204aa7c0, 0xc420d2b560, 0xc420d2b620) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4dc0, 0xc4204aa8c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 217 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4204aa7c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 205 [runnable]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldEnqueue(0xc421b4bb80, 0x1a1c020) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6594 +0x2a0 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4200e1900, 0xc42037db01, 0xc42037dba0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:298 +0x117 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4200e1900, 0xc420da7e00, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4200e1900, 0xc420da7e00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc4211100f0, 0xc4200e1900, 0xc420da7e00) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 206 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b4dc0, 0xc420f64d80, 0xc420d1a1e0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4dc0, 0xc4207ac0c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 207 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b4dc0, 0xc420f64e40, 0xc420d1ba40) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4dc0, 0xc4207ac240) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 226 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420598840) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 227 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b4dc0, 0xc420f64e40, 0x1d6eb80, 0xc420598840, 0xc420d62ae0, 0xc420d62cc0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4dc0, 0xc420598880) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 219 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420f3f2c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 209 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420473740, 0xc4207e8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc4216dbf80) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc4216dbf80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42172db90, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4205e80a0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420536780, 0x16e4b60, 0xc4205e8100) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420536780, 0x16e4b60, 0xc4205e8100, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4217233c0, 0x16e4b60, 0xc4205e8100, 0xc4205e8100, 0xc421126668) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4217233c0, 0xc4205e8100, 0x2d, 0xc422cf8860) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc421126640, 0x1d72340, 0xc4217233c0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42020aaf0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc421126640, 0x1d72340, 0xc4217233c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc421126640, 0x1d72340, 0xc4217233c0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc420473740, 0xc421126640) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 258 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).connect.func2(0x1d718c0, 0xc4227c2780, 0xc422c14100, 0xc4219121e0, 0xc421912180) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:129 +0xde Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by net.(*netFD).connect Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:128 +0x286 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 259 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422eddc30, 0xc420d6cd10, 0x30, 0xf9866a) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422eddc30, 0x1c61988, 0xc4200e1900, 0x1d71840, 0xc4210fa1c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc420211b10, 0xc423091d10) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d320, 0xc420211b10, 0xc423091d10) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 260 [IO wait]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.runtime_pollWait(0x7f9ff60cd200, 0x72, 0xc4214a1ce8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).wait(0xc420c91f98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*pollDesc).waitRead(0xc420c91f98, 0xc4209ba000, 0x1000, 0x1000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: internal/poll.(*FD).Read(0xc420c91f80, 0xc4209ba000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*netFD).Read(0xc420c91f80, 0xc4209ba000, 0x1000, 0x1000, 0xc4214a1dd8, 0x1, 0xc4214a1e08) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net.(*conn).Read(0xc420211b10, 0xc4209ba000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).Read(0xc422531320, 0xc4225993b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadAtLeast(0x1d59aa0, 0xc422531320, 0xc4225993b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: io.ReadFull(0x1d59aa0, 0xc422531320, 0xc4225993b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422eddc30, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422eddc30) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 261 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422eddc30) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 262 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422eddc30) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 263 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4232dc0c0, 0xc421121700, 0x20, 0x20, 0xc4232de000, 0x1e) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422acc990, 0x1d718c0, 0xc4232dc0c0, 0x0, 0xc4232dc000) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1900, 0xc421121950, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4226e4600, 0xc4221e8e80, 0xc422acc8a0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: reflect.Value.call(0xc4200872c0, 0xc420211e18, 0x13, 0x1bda9a0, 0x4, 0xc420d6fd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: reflect.Value.Call(0xc4200872c0, 0xc420211e18, 0x13, 0xc420d6fd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*service).call(0xc423114ac0, 0xc423091db0, 0xc4230ed060, 0x0, 0xc4231f5000, 0xc421f4d100, 0x19758e0, 0xc4221e8e80, 0x16, 0x169e020, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc423091db0, 0x1d72340, 0xc423114b00, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc42059afc0, 0xc423091db0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 264 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420f64780, 0xc4232d8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).fill(0xc4226e4fc0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: bufio.(*Reader).ReadByte(0xc4226e4fc0, 0x1c66260, 0xffffffffffffffff, 0xc420dafab0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422f566f0, 0x1) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421f4d280) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4226e5020, 0x16e4b60, 0xc421f4d320) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4226e5020, 0x16e4b60, 0xc421f4d320, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc423114c00, 0x16e4b60, 0xc421f4d320, 0xc421f4d320, 0xc423091dd8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc423114c00, 0xc421f4d320, 0x2d, 0xc422cf8da0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequestHeader(0xc423091db0, 0x1d72340, 0xc423114c00, 0x0, 0x2, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).readRequest(0xc423091db0, 0x1d72340, 0xc423114c00, 0x3, 0x3, 0xc420daff50, 0xc420dafe18, 0xb8, 0x6, 0xc420046f70, ...) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: net/rpc.(*Server).ServeRequest(0xc423091db0, 0x1d72340, 0xc423114c00, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1900, 0x1d71840, 0xc4210fa1c0, 0x1d7d0e0, 0xc420f64780, 0xc423091db0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 221 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc42020adc0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4200e1900) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 223 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420450c60) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 224 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420450c60) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 225 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc4226209f0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 274 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422b3df40, 0xc420564ce8, 0x20, 0x20, 0x18, 0xc4215001c0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421f261b0, 0x1d71840, 0xc422b3df40, 0xc4215001c0, 0x184987) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42037dba0, 0xc420dabed0, 0x184987, 0x1d71840, 0xc422b3df40, 0xc420f08960, 0xc4226209c0, 0xc420caa090, 0x24, 0x2bba400) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420f08900, 0x1d71840, 0xc422b3df40, 0x184987, 0xc422824040, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420f08900, 0x1d71840, 0xc422b3df40) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 275 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422b3dfc0, 0xc420e53770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422c7c000, 0x1d71840, 0xc422b3dfc0, 0x0, 0x1849b5) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42037dba0, 0xc420e53960, 0x1849b5, 0x1d71840, 0xc422b3dfc0, 0x552bde, 0xc420e53950, 0xc420e53970, 0xf33ba5, 0xc422f650f8) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc422f650a0, 0x1d71840, 0xc422b3dfc0, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc422f650a0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 276 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc421126460, 0x7f9ff6006390, 0xc422b3df80, 0x1, 0x0, 0x0) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:248 +0x2c1 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc421126460, 0x7f9ff6006390, 0xc422b3df80, 0x7f9ff6006390, 0xc422b3df80) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc421f24140, 0x1833a4, 0x3b, 0xc4214a2de8, 0x1, 0x1) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:214 +0x72 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc421f24140) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 277 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc421f24180) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: goroutine 278 [select]: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420ca05a0, 0x1d71840, 0xc422b3df80) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25967]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:28:49 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:28:49 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:28:49 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: ==> Starting Nomad agent... Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: ==> Nomad agent configuration: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: Client: false Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: Log Level: INFO Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: Region: amrv01 (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: Server: true Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: Version: 0.8.4 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.470559 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: Attempting re-join to previously known node: consulnomad-02.atf01: 172.29.0.72:4648 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.472770 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [WARN] memberlist: Refuting an alive message Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650100 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650124 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650421 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650440 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650452 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650465 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650502 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650520 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650830 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650847 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650857 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650868 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49 [INFO] serf: Re-joined to previously known node: consulnomad-02.atf01: 172.29.0.72:4648 2018/08/23 15:28:49.650880 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650896 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650908 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650918 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650929 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650941 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650952 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:28:49 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:49.650964 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:28:50 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:50 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:28:50 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:50 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 56 Aug 23 15:28:50 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:50 [INFO] raft: Election won. Tally: 2 Aug 23 15:28:50 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:50 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:28:50 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:50 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:28:50 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:50 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:28:50 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:50.940557 [INFO] nomad: cluster leadership acquired Aug 23 15:28:50 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:50 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:28:50 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:50 [INFO] raft: pipelining replication to peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:51 ip-172-24-20-236 nomad.sh[25988]: 2018/08/23 15:28:51 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: fatal error: concurrent map writes Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 180 [running]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc420ed3978 sp=0xc420ed3958 pc=0x42d161 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: runtime.mapassign(0x1844be0, 0xc420348c30, 0xc420ed3aa8, 0x2bba400) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/hashmap.go:519 +0x56e fp=0xc420ed3a08 sp=0xc420ed3978 pc=0x40a41e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc4203bff00, 0x1d6dd40, 0xc42000fe08, 0x1871a5bb, 0xed3108915, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:99 +0x1d1 fp=0xc420ed3af8 sp=0xc420ed3a08 pc=0xf20d91 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc420336f00, 0xc4207b18c0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc420ed3b60 sp=0xc420ed3af8 pc=0xf5388e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc420336f00, 0xc4207b18c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc420ed3b98 sp=0xc420ed3b60 pc=0xf534b3 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4200efb80, 0xc4203bfc01, 0xc4203bfc40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc420ed3c88 sp=0xc420ed3b98 pc=0xf77602 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4200efb80, 0xc420d9af00, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc420ed3d38 sp=0xc420ed3c88 pc=0xf76fff Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4200efb80, 0xc420d9af00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc420ed3fa0 sp=0xc420ed3d38 pc=0xf76b90 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc42100a090, 0xc4200efb80, 0xc420d9af00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc420ed3fc8 sp=0xc420ed3fa0 pc=0xfb8e9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: runtime.goexit() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420ed3fd0 sp=0xc420ed3fc8 pc=0x45c711 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 1 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420220d90, 0xc4203e7680) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420220d90, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4200fd680, 0xc4200fd680, 0xc420213400, 0x37) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204d7f68) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc42009e058) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: main.main() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 5 [syscall]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: os/signal.signal_recv(0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: os/signal.loop() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by os/signal.init.0 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 186 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421e8c000, 0xc420caf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).fill(0xc4230afe60) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).ReadByte(0xc4230afe60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422cf6480, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4229bcc00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4230afec0, 0x16e4b60, 0xc4229bcc60) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4230afec0, 0x16e4b60, 0xc4229bcc60, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422cd2580, 0x16e4b60, 0xc4229bcc60, 0xc4229bcc60, 0xc420ffa1b8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422cd2580, 0xc4229bcc60, 0x2d, 0xc423099040) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequestHeader(0xc420ffa190, 0x1d72340, 0xc422cd2580, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42043f540, 0x8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequest(0xc420ffa190, 0x1d72340, 0xc422cd2580, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc420ffa190, 0x1d72340, 0xc422cd2580, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc421e8c000, 0xc420ffa190) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 122 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc42043f810, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4200efb80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 111 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc42042b3c0, 0x1d71840, 0xc4204ae8c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 200 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4213749c0, 0xc420bc3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).fill(0xc420e32180) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).ReadByte(0xc420e32180, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d71a70, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207227c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420e321e0, 0x16e4b60, 0xc420722820) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420e321e0, 0x16e4b60, 0xc420722820, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4205a8940, 0x16e4b60, 0xc420722820, 0xc420722820, 0xc4214369d8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4205a8940, 0xc420722820, 0x2d, 0xc4207f9040) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequestHeader(0xc4214369b0, 0x1d72340, 0xc4205a8940, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42043f540, 0x8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequest(0xc4214369b0, 0x1d72340, 0xc4205a8940, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc4214369b0, 0x1d72340, 0xc4205a8940, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc4213749c0, 0xc4214369b0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 118 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420306300, 0xc42093e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).fill(0xc420446e40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).ReadByte(0xc420446e40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42098dec0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420bc7620) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420446ea0, 0x16e4b60, 0xc420bc7680) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420446ea0, 0x16e4b60, 0xc420bc7680, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4209de2c0, 0x16e4b60, 0xc420bc7680, 0xc420bc7680, 0xc420ffa1b8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4209de2c0, 0xc420bc7680, 0x2d, 0xc421e46f00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequestHeader(0xc420ffa190, 0x1d72340, 0xc4209de2c0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42043f540, 0x8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequest(0xc420ffa190, 0x1d72340, 0xc4209de2c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc420ffa190, 0x1d72340, 0xc4209de2c0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:52 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc420306300, 0xc420ffa190) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 198 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200c22c0, 0xc420150900, 0xc420ddfb00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc421011d40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 38 [select, locked to thread]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: runtime.selectgo(0xc420050f50, 0xc42009e780) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: runtime.ensureSigM.func1() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: runtime.goexit() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 39 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42042b300) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 40 [sleep]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: time.Sleep(0x3b9aca00) Aug 23 15:28:52 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc4202b1a20) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 41 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420306540) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 168 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420f22000, 0xc42055bd10, 0x30, 0xf9866a) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420f22000, 0x1c61988, 0xc4200efb80, 0x1d71840, 0xc420fef580) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d320, 0xc42000ed68, 0xc4210126e0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d320, 0xc42000ed68, 0xc4210126e0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 42 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4203c2f00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:28:52 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 121 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420f026e0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 79 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421374180, 0xc420eb4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).fill(0xc420318a80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).ReadByte(0xc420318a80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420c073e0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4212efd20) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420318ae0, 0x16e4b60, 0xc4212efd80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420318ae0, 0x16e4b60, 0xc4212efd80, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421360c00, 0x16e4b60, 0xc4212efd80, 0xc4212efd80, 0xc420ffa1b8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421360c00, 0xc4212efd80, 0x2d, 0xc421e46d80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequestHeader(0xc420ffa190, 0x1d72340, 0xc421360c00, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42043f540, 0x8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequest(0xc420ffa190, 0x1d72340, 0xc421360c00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc420ffa190, 0x1d72340, 0xc421360c00, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc421374180, 0xc420ffa190) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 44 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4200ebba0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0xc420270400, 0xc420270420) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc420349080, 0xc42020fe30) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 55 [IO wait]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.runtime_pollWait(0x7fe2c6352f00, 0x72, 0xc4200659a8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).wait(0xc420216598, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).waitRead(0xc420216598, 0xc420516000, 0x1000, 0x1000) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*FD).Read(0xc420216580, 0xc420516000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*netFD).Read(0xc420216580, 0xc420516000, 0x1000, 0x1000, 0x458600, 0xc42048f380, 0x4) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*conn).Read(0xc4202000f8, 0xc420516000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/http.(*persistConn).Read(0xc420380120, 0xc420516000, 0x1000, 0x1000, 0xc420065b98, 0x405675, 0xc42037a480) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).fill(0xc420318660) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).Peek(0xc420318660, 0x1, 0x0, 0x0, 0x1, 0xc420e5a900, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/http.(*persistConn).readLoop(0xc420380120) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by net/http.(*Transport).dialConn Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 56 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/http.(*persistConn).writeLoop(0xc420380120) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by net/http.(*Transport).dialConn Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 123 [semacquire]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: sync.runtime_SemacquireMutex(0xc420336f8c, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: sync.(*Mutex).Lock(0xc420336f88) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: sync.(*RWMutex).Lock(0xc420336f88) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc420336f00, 0x1d71840, 0xc422897940) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 201 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4227e1e00, 0xc422ba5d28, 0x20, 0x20, 0xc420056800, 0x1bd9a98) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4219bfda0, 0x1d71840, 0xc4227e1e00, 0xc421fb3c60, 0x1849b5) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203bfc40, 0xc422bb3f20, 0x1849b5, 0x1d71840, 0xc4227e1e00, 0x0, 0x0, 0x0, 0xc4203c2de0, 0x4) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc422d3a0a0, 0x1849b5, 0x0, 0x0, 0xc4213577c8, 0xfb15f8, 0xc421001da0, 0x1dcd6500) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc422d3a0a0, 0x1849b5, 0xc42140d560) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 170 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200c22c0, 0xc420150900, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200c22c0, 0xc420150900) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc420ad2d60) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 74 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc42043fa40, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4200cc2c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 176 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420f220b0, 0xc421367d10, 0x30, 0xf9866a) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420f220b0, 0x1c61988, 0xc4200efb80, 0x1d71840, 0xc420fef580) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d320, 0xc4202000d0, 0xc4205ec690) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d320, 0xc4202000d0, 0xc4205ec690) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 78 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420f22000) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 80 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421374240, 0xc420e4c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).fill(0xc420319080) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).ReadByte(0xc420319080, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420bb82a0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4212eff40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4203190e0, 0x16e4b60, 0xc4212effa0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4203190e0, 0x16e4b60, 0xc4212effa0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421360d40, 0x16e4b60, 0xc4212effa0, 0xc4212effa0, 0xc420ffa1b8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421360d40, 0xc4212effa0, 0x2d, 0xc421e46fe0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequestHeader(0xc420ffa190, 0x1d72340, 0xc421360d40, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42043f540, 0x8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequest(0xc420ffa190, 0x1d72340, 0xc421360d40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc420ffa190, 0x1d72340, 0xc421360d40, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc421374240, 0xc420ffa190) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 77 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420f22000) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 130 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200c22c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200c22c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200c22c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc42073ab70) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 117 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200c22c0, 0xc4201509c0, 0xc420d680c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc420bc7200) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 131 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200c22c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc42073ab80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 132 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200c22c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc42073ab90) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 133 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420ffef30) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 134 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4204245b0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 135 [IO wait]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.runtime_pollWait(0x7fe2c6352d60, 0x72, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).wait(0xc4203a0398, 0x72, 0xc4202b8200, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).waitRead(0xc4203a0398, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*FD).Accept(0xc4203a0380, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*netFD).accept(0xc4203a0380, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*TCPListener).accept(0xc42000e630, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*TCPListener).AcceptTCP(0xc42000e630, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420cb83c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 136 [IO wait]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.runtime_pollWait(0x7fe2c6352c90, 0x72, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).wait(0xc4203a0618, 0x72, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).waitRead(0xc4203a0618, 0xc42238e000, 0x10000, 0x10000) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*FD).ReadFrom(0xc4203a0600, 0xc42238e000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*netFD).readFrom(0xc4203a0600, 0xc42238e000, 0x10000, 0x10000, 0xeb, 0xeb, 0x1d62b40, 0x412539, 0xc420063e40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*UDPConn).readFrom(0xc42000e638, 0xc42238e000, 0x10000, 0x10000, 0x101, 0xc420063e18, 0xc420063e90, 0x443cf7) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*UDPConn).ReadFrom(0xc42000e638, 0xc42238e000, 0x10000, 0x10000, 0x10000, 0x10000, 0x681b, 0x92bd66e7, 0x2b96de0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420cb83c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 137 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420cb83c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 138 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420cb83c0, 0x12a05f200, 0xc421001c80, 0xc420ee0a20, 0xc42073b1d0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:110 +0xef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 139 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420cb83c0, 0xc420ee0a20) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 140 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420cb83c0, 0x1dcd6500, 0xc421001ce0, 0xc420ee0a20, 0xc42073b1f0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 141 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc42056d200) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 142 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc42056d200) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 143 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42056d200, 0x1bdd09a, 0x6, 0xc420fff080) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 144 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42056d200, 0x1bdba6c, 0x5, 0xc420fff0b0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 145 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc42056d200, 0x1bdbc6a, 0x5, 0xc420fff0e0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 76 [IO wait]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.runtime_pollWait(0x7fe2c6352bc0, 0x72, 0xc421363ce8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).wait(0xc4209d5e18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).waitRead(0xc4209d5e18, 0xc420eaf000, 0x1000, 0x1000) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*FD).Read(0xc4209d5e00, 0xc420eaf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*netFD).Read(0xc4209d5e00, 0xc420eaf000, 0x1000, 0x1000, 0x8e, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*conn).Read(0xc42000ed68, 0xc420eaf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).Read(0xc42136f3e0, 0xc42100aee0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: io.ReadAtLeast(0x1d59aa0, 0xc42136f3e0, 0xc42100aee0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: io.ReadFull(0x1d59aa0, 0xc42136f3e0, 0xc42100aee0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420f22000, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420f22000) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 147 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421001d40, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421001d40, 0x1dcd6500, 0x72617453aa006c61, 0xaa0079616c654474, 0x6e6769536b736154, 0x6b736154b0a06c61, 0x65526c616e676953) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421001d40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 148 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421001da0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421001da0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421001da0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 149 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421001e00, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421001e00, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421001e00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 150 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421001e60, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421001e60, 0x1dcd6500, 0xc4202e11e0, 0xc4202e1200, 0xc4202e1220, 0xc4202e1240, 0xc4202e1260) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421001e60) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 151 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc4212ed0e0, 0xc4200efb80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 152 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4200efb80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 153 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4200efb80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 154 [IO wait]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.runtime_pollWait(0x7fe2c6352e30, 0x72, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).wait(0xc420598518, 0x72, 0xc420083900, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).waitRead(0xc420598518, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*FD).Accept(0xc420598500, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*netFD).accept(0xc420598500, 0x1, 0x1, 0xc420062f20) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*TCPListener).accept(0xc4204a71c0, 0x406b4a, 0xc420f6fec0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*TCPListener).Accept(0xc4204a71c0, 0xc420f6fec0, 0x0, 0x3f800000, 0xc420fef580) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4200efb80, 0x1d71840, 0xc420fef580) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 155 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc420336f00, 0x3b9aca00, 0xc42009ec00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 156 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc42043f810, 0x3b9aca00, 0xc42009ec00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 157 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc4202babd0, 0x3b9aca00, 0xc42009ec00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 158 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4200ebba0, 0x3b9aca00, 0xc42009ec00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 159 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4200efb80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 160 [IO wait]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.runtime_pollWait(0x7fe2c6352af0, 0x72, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).wait(0xc420599698, 0x72, 0xc421360100, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).waitRead(0xc420599698, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*FD).Accept(0xc420599680, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*netFD).accept(0xc420599680, 0x402037, 0xc42029dca0, 0x1805ec0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*TCPListener).accept(0xc42000e960, 0x458e40, 0xc42137fe90, 0xc42137fe98) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*TCPListener).Accept(0xc42000e960, 0x1c65af8, 0xc42029dc20, 0x1d71900, 0xc42135c150) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/http.(*Server).Serve(0xc4204ef860, 0x1d6fc00, 0xc42000e960, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/http.Serve(0x1d6fc00, 0xc42000e960, 0x1d5df00, 0xc4212ed350, 0x7865646e49796669, 0x654ca9597e0e00ce) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420fef640, 0x1d6fc00, 0xc42000e960, 0xc42073b800, 0xc4212ed1a0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 161 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420599900, 0xc42073b810, 0xc420ee1860) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 162 [sleep]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: time.Sleep(0x2d399f075) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc420220d90, 0xc420599900) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 163 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202b14a0, 0xc421378d10, 0x30, 0xf9866a) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202b14a0, 0x1c61988, 0xc4200efb80, 0x1d71840, 0xc420fef580) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d320, 0xc42000e4b8, 0xc421012370) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d320, 0xc42000e4b8, 0xc421012370) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 164 [IO wait]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.runtime_pollWait(0x7fe2c6352a20, 0x72, 0xc42137dce8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).wait(0xc420599a18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).waitRead(0xc420599a18, 0xc42100d000, 0x1000, 0x1000) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*FD).Read(0xc420599a00, 0xc42100d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*netFD).Read(0xc420599a00, 0xc42100d000, 0x1000, 0x1000, 0xc420edbdd8, 0x1, 0xc420edbe08) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*conn).Read(0xc42000e4b8, 0xc42100d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).Read(0xc42100eb40, 0xc4213580a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: io.ReadAtLeast(0x1d59aa0, 0xc42100eb40, 0xc4213580a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: io.ReadFull(0x1d59aa0, 0xc42100eb40, 0xc4213580a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202b14a0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202b14a0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 165 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202b14a0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 166 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202b14a0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 167 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420306000, 0xc421260000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).fill(0xc421001140) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).ReadByte(0xc421001140, 0x100000000000016, 0x0, 0x6a0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420ffe900, 0x413e07) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4210101a0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210011a0, 0x16e4b60, 0xc421010200) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210011a0, 0x16e4b60, 0xc421010200, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420fee440, 0x16e4b60, 0xc421010200, 0xc421010200, 0xc421012488) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420fee440, 0xc421010200, 0x2d, 0xc4207c2c80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequestHeader(0xc421012460, 0x1d72340, 0xc420fee440, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42043f540, 0x8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequest(0xc421012460, 0x1d72340, 0xc420fee440, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc421012460, 0x1d72340, 0xc420fee440, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc420306000, 0xc421012460) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 169 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420150840, 0xc420c6e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).fill(0xc421432e40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).ReadByte(0xc421432e40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420b74750, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420ad2760) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421432ea0, 0x16e4b60, 0xc420ad27e0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421432ea0, 0x16e4b60, 0xc420ad27e0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421361280, 0x16e4b60, 0xc420ad27e0, 0xc420ad27e0, 0xc420ffa1b8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421361280, 0xc420ad27e0, 0x2d, 0xc422966740) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequestHeader(0xc420ffa190, 0x1d72340, 0xc421361280, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42043f540, 0x8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequest(0xc420ffa190, 0x1d72340, 0xc421361280, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc420ffa190, 0x1d72340, 0xc421361280, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc420150840, 0xc420ffa190) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 171 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200c22c0, 0xc4201509c0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200c22c0, 0xc4201509c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc420ad2d80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 172 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc421361640) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 173 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200c22c0, 0xc4201509c0, 0x1d6eb80, 0xc421361640, 0xc420d7b4a0, 0xc420d7b560) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc421361680) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 174 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4213617c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 175 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200c22c0, 0xc420150900, 0x1d6eb80, 0xc4213617c0, 0xc420d7baa0, 0xc420d561e0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200c22c0, 0xc421361800) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 181 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42014e370, 0xc420c68d10, 0x30, 0xf9866a) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42014e370, 0x1c61988, 0xc4200efb80, 0x1d71840, 0xc420fef580) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d320, 0xc420200258, 0xc4214368c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d320, 0xc420200258, 0xc4214368c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 182 [IO wait]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.runtime_pollWait(0x7fe2c6352610, 0x72, 0xc42137ace8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).wait(0xc420baa218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).waitRead(0xc420baa218, 0xc420874000, 0x1000, 0x1000) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*FD).Read(0xc420baa200, 0xc420874000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*netFD).Read(0xc420baa200, 0xc420874000, 0x1000, 0x1000, 0xc42137add8, 0x1, 0xc42137ae08) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*conn).Read(0xc420200258, 0xc420874000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).Read(0xc420086ba0, 0xc420b16060, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: io.ReadAtLeast(0x1d59aa0, 0xc420086ba0, 0xc420b16060, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: io.ReadFull(0x1d59aa0, 0xc420086ba0, 0xc420b16060, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42014e370, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42014e370) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 183 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42014e370) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 184 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42014e370) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 185 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420e3c720, 0xc421385700, 0x20, 0x20, 0x169d4e0, 0xc4203bfe60) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420ea1d40, 0x1d718c0, 0xc420e3c720, 0x0, 0xc420e3c700) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200efb80, 0xc421385950, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420acc960, 0xc420ec9580, 0xc420ea1b60, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: reflect.Value.call(0xc4205631a0, 0xc420200980, 0x13, 0x1bda9a0, 0x4, 0xc421385d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: reflect.Value.Call(0xc4205631a0, 0xc420200980, 0x13, 0xc421385d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*service).call(0xc420fefa00, 0xc4214369b0, 0xc4212ab7d8, 0x0, 0xc420baeb00, 0xc420ff4b60, 0x19758e0, 0xc420ec9580, 0x16, 0x169e020, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc4214369b0, 0x1d72340, 0xc420fefa80, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc420150a80, 0xc4214369b0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 177 [IO wait]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.runtime_pollWait(0x7fe2c6352950, 0x72, 0xc42137bce8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).wait(0xc420c0c118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).waitRead(0xc420c0c118, 0xc421261000, 0x1000, 0x1000) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*FD).Read(0xc420c0c100, 0xc421261000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*netFD).Read(0xc420c0c100, 0xc421261000, 0x1000, 0x1000, 0x2f, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*conn).Read(0xc4202000d0, 0xc421261000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).Read(0xc421041140, 0xc420730470, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: io.ReadAtLeast(0x1d59aa0, 0xc421041140, 0xc420730470, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: io.ReadFull(0x1d59aa0, 0xc421041140, 0xc420730470, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420f220b0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420f220b0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 210 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420f220b0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 211 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420f220b0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 212 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420306180, 0xc420c9f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).fill(0xc420299260) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).ReadByte(0xc420299260, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42020ccf0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421222aa0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4202992c0, 0x16e4b60, 0xc421222b00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4202992c0, 0x16e4b60, 0xc421222b00, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42148ce80, 0x16e4b60, 0xc421222b00, 0xc421222b00, 0xc4205ec758) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42148ce80, 0xc421222b00, 0x2d, 0xc421bfe8a0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequestHeader(0xc4205ec730, 0x1d72340, 0xc42148ce80, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42043f540, 0x8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequest(0xc4205ec730, 0x1d72340, 0xc42148ce80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc4205ec730, 0x1d72340, 0xc42148ce80, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc420306180, 0xc4205ec730) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 120 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420f026e0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 187 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420d580b0, 0xc420c62d10, 0x30, 0xf9866a) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420d580b0, 0x1c61988, 0xc4200efb80, 0x1d71840, 0xc420fef580) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d320, 0xc420200208, 0xc420d3b6d0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d320, 0xc420200208, 0xc420d3b6d0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 214 [IO wait]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.runtime_pollWait(0x7fe2c6352470, 0x72, 0xc420e65ce8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).wait(0xc42223c298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*pollDesc).waitRead(0xc42223c298, 0xc422af4000, 0x1000, 0x1000) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: internal/poll.(*FD).Read(0xc42223c280, 0xc422af4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*netFD).Read(0xc42223c280, 0xc422af4000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net.(*conn).Read(0xc420200208, 0xc422af4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).Read(0xc422f11980, 0xc420f93cc0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: io.ReadAtLeast(0x1d59aa0, 0xc422f11980, 0xc420f93cc0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: io.ReadFull(0x1d59aa0, 0xc422f11980, 0xc420f93cc0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420d580b0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420d580b0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 215 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420d580b0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 216 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420d580b0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 188 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421a57260, 0xc42038d700, 0x20, 0x20, 0x169d4e0, 0xc4200116c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421f193e0, 0x1d718c0, 0xc421a57260, 0x0, 0xc421a57200) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200efb80, 0xc42038d950, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420e32840, 0xc42223c380, 0xc421f19230, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: reflect.Value.call(0xc4205631a0, 0xc420210d48, 0x13, 0x1bda9a0, 0x4, 0xc420e69d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: reflect.Value.Call(0xc4205631a0, 0xc420210d48, 0x13, 0xc420e69d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*service).call(0xc4227e1940, 0xc421fbeb40, 0xc4221fdb18, 0x0, 0xc4210c0a00, 0xc421fb37e0, 0x19758e0, 0xc42223c380, 0x16, 0x169e020, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc421fbeb40, 0x1d72340, 0xc421efa280, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc421e8c0c0, 0xc421fbeb40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 189 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc421375740, 0xc42117b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).fill(0xc421a573e0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: bufio.(*Reader).ReadByte(0xc421a573e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421f197a0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421fb39a0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421a57440, 0x16e4b60, 0xc421fb3a20) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421a57440, 0x16e4b60, 0xc421fb3a20, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421efaa40, 0x16e4b60, 0xc421fb3a20, 0xc421fb3a20, 0xc421fbeb68) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421efaa40, 0xc421fb3a20, 0x2d, 0xc4229666e0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequestHeader(0xc421fbeb40, 0x1d72340, 0xc421efaa40, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42043f540, 0x8) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).readRequest(0xc421fbeb40, 0x1d72340, 0xc421efaa40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: net/rpc.(*Server).ServeRequest(0xc421fbeb40, 0x1d72340, 0xc421efaa40, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200efb80, 0x1d71840, 0xc420fef580, 0x1d7d0e0, 0xc421375740, 0xc421fbeb40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 124 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc4202babd0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 125 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc4202babd0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 126 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc4227b38c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 127 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422897980, 0xc420e69ce8, 0x20, 0x20, 0x18, 0xc422885480) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421ea91d0, 0x1d71840, 0xc422897980, 0xc422885480, 0x184987) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203bfc40, 0xc420ce9ed0, 0x184987, 0x1d71840, 0xc422897980, 0xc4202d6360, 0xc4227b3890, 0xc420c27740, 0x24, 0x2bba400) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc4202d6300, 0x1d71840, 0xc422897980, 0x184987, 0xc4227e1d00, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc4202d6300, 0x1d71840, 0xc422897980) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 128 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422897a00, 0xc42109b770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4219bfe30, 0x1d71840, 0xc422897a00, 0x0, 0x1849b5) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203bfc40, 0xc42109b960, 0x1849b5, 0x1d71840, 0xc422897a00, 0x552bde, 0xc42109b950, 0xc42109b970, 0xf33ba5, 0xc4202da368) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc4202da310, 0x1d71840, 0xc422897a00, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc4202da310) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 129 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4228979c0, 0xc421097a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421f198f0, 0x1d71840, 0xc4228979c0, 0xc421f19aa0, 0x1833a4) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203bfc40, 0xc421097c68, 0x1833a4, 0x1d71840, 0xc4228979c0, 0x552bde, 0xc42043f5e0, 0x2, 0xc42101be40, 0x40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422897a40, 0x1833a4, 0x3b, 0xc422babde8, 0x1, 0x1) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422897a40) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 226 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422897a80) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 227 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420cb85a0, 0x1d71840, 0xc4228979c0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 190 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc421f197d0, 0xc421982840) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 191 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc421f197d0, 0xc421982960) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: goroutine 217 [select]: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc422d3a0a0) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[25988]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:28:52 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:28:52 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:28:52 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: ==> Starting Nomad agent... Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: ==> Nomad agent configuration: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: Client: false Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: Log Level: INFO Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: Region: amrv01 (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: Server: true Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: Version: 0.8.4 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.721275 [INFO] nomad: starting 4 scheduling worker(s) for [system service batch _core] Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.721357 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: Attempting re-join to previously known node: ip-172-29-4-194.atf01: 172.29.4.194:4648 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899548 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899570 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899584 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899595 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899613 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899625 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899901 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899922 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899934 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899946 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899961 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.899972 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [WARN] memberlist: Refuting an alive message Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52 [INFO] serf: Re-joined to previously known node: ip-172-29-4-194.atf01: 172.29.4.194:4648 Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.900574 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.900586 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.900596 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.900607 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.900617 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.900627 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.900636 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:28:52 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:52.900647 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:54 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:54 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:28:56 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:56 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:28:56 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:56 [WARN] raft: Rejecting vote request from 172.24.0.121:4647 since we have a leader: 172.24.10.89:4647 Aug 23 15:28:56 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:56 [WARN] raft: Failed to get previous log: 1598291 log not found (last: 1598289) Aug 23 15:28:58 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:28:58.965585 [INFO] server.nomad: successfully contacted 20 Nomad Servers Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05 [ERR] raft-net: Failed to decode incoming command: read tcp 172.24.20.236:4647->172.24.0.121:52210: read: connection reset by peer Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.648536 [ERR] worker: failed to update evaluation : rpc error: stream closed Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.648595 [ERR] worker: failed to nack evaluation '22f95171-d8ae-de56-0e1b-d7cfa8c7b336': rpc error: stream closed Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.648631 [ERR] worker: failed to dequeue evaluation: rpc error: stream closed Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.648749 [ERR] worker: failed to update evaluation : rpc error: stream closed Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.648780 [ERR] worker: failed to nack evaluation '22f95171-d8ae-de56-0e1b-d7cfa8c7b336': rpc error: stream closed Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.649655 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.649672 [ERR] worker: failed to update evaluation : rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.649697 [ERR] worker: failed to update evaluation : rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.650584 [ERR] worker: failed to nack evaluation '22f95171-d8ae-de56-0e1b-d7cfa8c7b336': rpc error: failed to get conn: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.650667 [ERR] worker: failed to nack evaluation '22f95171-d8ae-de56-0e1b-d7cfa8c7b336': rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.651412 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.651452 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.669795 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.670707 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.672463 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.672500 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.750957 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.751009 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.753538 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: dial tcp 172.24.0.121:4647: connect: connection refused Aug 23 15:29:05 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:05.753570 [ERR] worker: failed to dequeue evaluation: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:06 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:06 [INFO] serf: EventMemberUpdate: ip-172-24-0-121.amrv01 Aug 23 15:29:07 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:07 [WARN] raft: Rejecting vote request from 172.24.10.89:4647 since we have a leader: 172.24.0.121:4647 Aug 23 15:29:07 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:07.291765 [ERR] worker: failed to dequeue evaluation: rpc error: rpc error: eval broker disabled Aug 23 15:29:07 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:07.378746 [ERR] worker: failed to dequeue evaluation: rpc error: rpc error: failed to get conn: dial tcp 172.24.10.89:4647: connect: connection refused Aug 23 15:29:07 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:07.391188 [ERR] worker: failed to dequeue evaluation: rpc error: rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:07 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:07.391230 [ERR] worker: failed to dequeue evaluation: rpc error: rpc error: failed to get conn: dial tcp 172.24.10.89:4647: connect: connection refused Aug 23 15:29:09 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:09.231158 [ERR] worker: failed to dequeue evaluation: rpc error: eval broker disabled Aug 23 15:29:09 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:09.240257 [ERR] worker: failed to dequeue evaluation: rpc error: eval broker disabled Aug 23 15:29:09 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:09.287943 [ERR] worker: failed to dequeue evaluation: rpc error: eval broker disabled Aug 23 15:29:09 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:09.438724 [ERR] worker: failed to dequeue evaluation: rpc error: eval broker disabled Aug 23 15:29:09 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:09 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:29:11 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:11 [WARN] raft: Rejecting vote request from 172.24.0.121:4647 since we have a leader: 172.24.10.89:4647 Aug 23 15:29:11 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:11.259965 [ERR] http: Request /v1/agent/health?type=server, error: {"server":{"ok":false,"message":"rpc error: rpc error: EOF"}} Aug 23 15:29:11 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:11 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:12 [WARN] raft: Heartbeat timeout from "172.24.0.121:4647" reached, starting election Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:12 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 62 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:12 [INFO] raft: Election won. Tally: 2 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:12 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:12 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:12 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:12.335603 [INFO] nomad: cluster leadership acquired Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:12 [WARN] raft: AppendEntries to {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} rejected, sending older logs (next: 1598444) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: 2018/08/23 15:29:12 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: fatal error: concurrent map writes Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 441 [running]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc4238d3d08 sp=0xc4238d3ce8 pc=0x42d161 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: runtime.mapdelete(0x1844be0, 0xc42020d890, 0xc4238d3e20) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc4238d3d68 sp=0xc4238d3d08 pc=0x40a761 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc4202c4ac0, 0x1d6dd40, 0xc42114d410, 0x0, 0xed3109f98) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc4238d3e50 sp=0xc4238d3d68 pc=0xf2164c Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4204f0000, 0x1d71840, 0xc422436780) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc4238d3fc8 sp=0xc4238d3e50 pc=0xf56bb7 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: runtime.goexit() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc4238d3fd0 sp=0xc4238d3fc8 pc=0x45c711 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 1 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42031dea0, 0xc420498780) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42031dea0, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4204ee780, 0xc4204ee780, 0xc420377700, 0x37) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204dff68) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: main.main() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 5 [syscall]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: os/signal.signal_recv(0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: os/signal.loop() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by os/signal.init.0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 358 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4225ac000, 0xc422edcd10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4225ac000, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42114c5e8, 0xc420e68d70) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42114c5e8, 0xc420e68d70) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 255 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200d6b00, 0xc421273d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200d6b00, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42000f9b0, 0xc4232de9b0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42000f9b0, 0xc4232de9b0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 167 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30950, 0x72, 0xc4212f4ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc42098d218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc42098d218, 0xc420afa000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc42098d200, 0xc420afa000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc42098d200, 0xc420afa000, 0x1000, 0x1000, 0xc4212f4dd8, 0x0, 0xc4212f4e08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc42000ed20, 0xc420afa000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc4208de0c0, 0xc42061f7b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc4208de0c0, 0xc42061f7b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc4208de0c0, 0xc42061f7b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420072160, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420072160) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 37 [select, locked to thread]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: runtime.selectgo(0xc4204bcf50, 0xc4204020c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: runtime.ensureSigM.func1() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: runtime.goexit() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 9 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc4201fe040) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 10 [sleep]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: time.Sleep(0x3b9aca00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc4200720b0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 11 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc4200743c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 68 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc420210140, 0x1d71840, 0xc4204fe140) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 60 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200d6420) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 357 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420073ce0, 0xc4212d3d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420073ce0, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420217940, 0xc420e68c30) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420217940, 0xc420e68c30) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 12 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc420228360) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 409 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92e618, 0x72, 0xc420060ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420dacc18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420dacc18, 0xc42247a000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc420dacc00, 0xc42247a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc420dacc00, 0xc42247a000, 0x1000, 0x1000, 0xc420060dd8, 0x0, 0xc420060e08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420503f78, 0xc42247a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc42050ea80, 0xc42052fbc0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc42050ea80, 0xc42052fbc0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc42050ea80, 0xc42052fbc0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420caf760, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420caf760) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 14 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4200a0750) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc42033c210, 0xc42020e380) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 66 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30f00, 0x72, 0xc4200669a8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc42044e598, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc42044e598, 0xc420474000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc42044e580, 0xc420474000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc42044e580, 0xc420474000, 0x1000, 0x1000, 0x458600, 0xc4204aa780, 0x4) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420502020, 0xc420474000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*persistConn).Read(0xc4200cc5a0, 0xc420474000, 0x1000, 0x1000, 0xc420066b98, 0x405675, 0xc4200906c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc42050ea20) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Peek(0xc42050ea20, 0x1, 0x0, 0x0, 0x1, 0xc420dde060, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*persistConn).readLoop(0xc4200cc5a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by net/http.(*Transport).dialConn Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 67 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*persistConn).writeLoop(0xc4200cc5a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by net/http.(*Transport).dialConn Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 58 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200d6420, 0xc4204dad10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200d6420, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420502008, 0xc420948140) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420502008, 0xc420948140) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 93 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4203bc000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203bc000, 0xc4205497d0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 334 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92ea28, 0x72, 0xc422edbce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc422376f18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc422376f18, 0xc42186c000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc422376f00, 0xc42186c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc422376f00, 0xc42186c000, 0x1000, 0x1000, 0xc422edbdd8, 0x0, 0xc422edbe08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420205430, 0xc42186c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc42128a6c0, 0xc420ff6a80, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc42128a6c0, 0xc420ff6a80, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc42128a6c0, 0xc420ff6a80, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421c2c840, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421c2c840) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 92 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4203bc000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203bc000, 0xc4205497c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 335 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421c2c840) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 59 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30a20, 0x72, 0xc4204dbce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420e38498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420e38498, 0xc4208a2000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc420e38480, 0xc4208a2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc420e38480, 0xc4208a2000, 0x1000, 0x1000, 0xc4204dbdd8, 0x0, 0xc4204dbe08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420502008, 0xc4208a2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc420946120, 0xc4206660c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc420946120, 0xc4206660c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc420946120, 0xc4206660c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200d6420, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200d6420) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 15 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc42020a870, 0x1c636a0, 0xc420072210, 0x1d71840, 0xc4203fb400) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc420072210) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 61 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200d6420) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 346 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30bc0, 0x72, 0xc4212d4ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc422376798, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc422376798, 0xc422967000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc422376780, 0xc422967000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc422376780, 0xc422967000, 0x1000, 0x1000, 0xc4212d4dd8, 0x0, 0xc4212d4e08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420217920, 0xc422967000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc420883f80, 0xc421c634a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc420883f80, 0xc421c634a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc420883f80, 0xc421c634a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421c2c6e0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421c2c6e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 91 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4203bc000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4203bc000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4203bc000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:29:12 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203bc000, 0xc4205497b0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 94 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420b76030) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 440 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc42020a500, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc420422000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 385 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42293daa0, 0xc4239ff700, 0x20, 0x20, 0x169d4e0, 0xc4218d1680) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42186b1a0, 0x1d718c0, 0xc42293daa0, 0x0, 0xc42293da00) Aug 23 15:29:12 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420422000, 0xc4239ff950, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4224893e0, 0xc42266c380, 0xc42186af60, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc420086900, 0xc420205eb0, 0x13, 0x1bda9a0, 0x4, 0xc42303fd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc420086900, 0xc420205eb0, 0x13, 0xc42303fd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc422109640, 0xc420e41540, 0xc42034f518, 0x0, 0xc421569700, 0xc420fe48c0, 0x19758e0, 0xc42266c380, 0x16, 0x169e020, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc420e41540, 0x1d72340, 0xc421ac5900, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cde00, 0xc420e41540) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 337 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4220e62a0, 0xc4239f9700, 0x20, 0x20, 0x169d4e0, 0xc4218d1680) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4210cf560, 0x1d718c0, 0xc4220e62a0, 0x0, 0xc4220e6200) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420422000, 0xc4239f9950, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420e6f440, 0xc4219bdc80, 0xc4210cf0e0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc420086900, 0xc4202057f0, 0x13, 0x1bda9a0, 0x4, 0xc4231cdd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc420086900, 0xc4202057f0, 0x13, 0xc4231cdd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc421cabf00, 0xc420e40f50, 0xc420d49538, 0x0, 0xc4219bd880, 0xc420a8fd60, 0x19758e0, 0xc4219bdc80, 0x16, 0x169e020, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc420e40f50, 0x1d72340, 0xc421cabf40, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cd740, 0xc420e40f50) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 205 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92f0a8, 0x72, 0xc421476ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420f43398, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420f43398, 0xc42130b000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc420f43380, 0xc42130b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc420f43380, 0xc42130b000, 0x1000, 0x1000, 0xfc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc42114cc48, 0xc42130b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc421be1bc0, 0xc42247c3a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc421be1bc0, 0xc42247c3a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc421be1bc0, 0xc42247c3a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420cae580, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420cae580) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 95 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420dd81a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 96 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30d60, 0x72, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420902098, 0x72, 0xc420082000, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420902098, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Accept(0xc420902080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).accept(0xc420902080, 0xc4226100c0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*TCPListener).accept(0xc420216298, 0x458e40, 0xc4204bd790, 0xc4204bd798) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*TCPListener).AcceptTCP(0xc420216298, 0x1c63658, 0xc420b663c0, 0xc4226100c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420b663c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 97 [runnable]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: syscall.Syscall6(0x2d, 0x9, 0xc423a00000, 0x10000, 0x0, 0xc4230a9c18, 0xc4230a9c0c, 0x10b, 0x10000, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: syscall.recvfrom(0x9, 0xc423a00000, 0x10000, 0x10000, 0x0, 0xc4230a9c18, 0xc4230a9c0c, 0x7f4bfba30cb8, 0xc4230a9c20, 0x4297c7) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/syscall/zsyscall_linux_amd64.go:1665 +0xa6 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: syscall.Recvfrom(0x9, 0xc423a00000, 0x10000, 0x10000, 0x0, 0xffffffffffffffff, 0x0, 0x0, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/syscall/syscall_unix.go:252 +0xaf Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).ReadFrom(0xc420902100, 0xc423a00000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:203 +0xff Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).readFrom(0xc420902100, 0xc423a00000, 0x10000, 0x10000, 0xe9, 0xe9, 0x1d62b40, 0x412539, 0xc4230a9e40) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*UDPConn).readFrom(0xc4202162f8, 0xc423a00000, 0x10000, 0x10000, 0x101, 0xc4230a9e18, 0xc4230a9e90, 0x443cf7) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*UDPConn).ReadFrom(0xc4202162f8, 0xc423a00000, 0x10000, 0x10000, 0x10000, 0x10000, 0xc9ec, 0x49cecb33c, 0x2b96de0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420b663c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 130 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420b663c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 131 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420b663c0, 0x12a05f200, 0xc420e6f920, 0xc420e2bb00, 0xc420549e10) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 132 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420b663c0, 0xc420e2bb00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 133 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420b663c0, 0x1dcd6500, 0xc420e6f980, 0xc420e2bb00, 0xc420549e20) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 134 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4202f9b00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 135 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4202f9b00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 136 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202f9b00, 0x1bdd09a, 0x6, 0xc420b76180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 137 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202f9b00, 0x1bdba6c, 0x5, 0xc420b761b0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 138 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202f9b00, 0x1bdbc6a, 0x5, 0xc420b761e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 180 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42023a9a0, 0xc4212f5d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42023a9a0, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420217168, 0xc420e681e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420217168, 0xc420e681e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 140 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e6f9e0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e6f9e0, 0x1dcd6500, 0x24, 0xc42213d680, 0x24, 0xc42065af00, 0xc420cc5400) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e6f9e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 141 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e6fa40, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e6fa40, 0x1dcd6500, 0x24, 0xc4221c0510, 0x24, 0xc42065b300, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e6fa40) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 142 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e6faa0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e6faa0, 0x1dcd6500, 0x24, 0xc4221c0270, 0x24, 0xc421173900, 0xc42021c500) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e6faa0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 143 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e6fb00, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e6fb00, 0x1dcd6500, 0x24, 0xc42213d1d0, 0x24, 0xc42065aa00, 0x73616c4364657400) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e6fb00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 144 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420b541e0, 0xc420422000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 145 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc420422000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 146 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc420422000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 147 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30e30, 0x72, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc4200f1218, 0x72, 0xc4208b8000, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc4200f1218, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Accept(0xc4200f1200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).accept(0xc4200f1200, 0x1, 0x1, 0xc4204d9f20) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*TCPListener).accept(0xc420216eb0, 0x406b4a, 0xc420e58000, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*TCPListener).Accept(0xc420216eb0, 0xc420e58000, 0x0, 0x3f800000, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 148 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4204f0000, 0x3b9aca00, 0xc4202ec540) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 149 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc42020a500, 0x3b9aca00, 0xc4202ec540) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 150 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420256480, 0x3b9aca00, 0xc4202ec540) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 151 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4200a0750, 0x3b9aca00, 0xc4202ec540) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 152 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc420422000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 153 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30af0, 0x72, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420903f18, 0x72, 0xc4201fe100, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420903f18, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Accept(0xc420903f00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).accept(0xc420903f00, 0x402037, 0xc4206e7840, 0x1805ec0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*TCPListener).accept(0xc420216dc8, 0x458e40, 0xc4212f6e90, 0xc4212f6e98) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*TCPListener).Accept(0xc420216dc8, 0x1c65af8, 0xc4206e77c0, 0x1d71900, 0xc420e0f770) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*Server).Serve(0xc4211892b0, 0x1d6fc00, 0xc420216dc8, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.Serve(0x1d6fc00, 0xc420216dc8, 0x1d5df00, 0xc420e0f6b0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420b52240, 0x1d6fc00, 0xc420216dc8, 0xc4201fc410, 0xc420b542a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 154 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc4200f1700, 0xc4201fc420, 0xc420e2bf20) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 392 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42096d440, 0xc42150a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc422009080) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc422009080, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4218e4150, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420fe4b60) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4220090e0, 0x16e4b60, 0xc420fe4bc0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4220090e0, 0x16e4b60, 0xc420fe4bc0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421ac5e00, 0x16e4b60, 0xc420fe4bc0, 0xc420fe4bc0, 0xc420e41568) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421ac5e00, 0xc420fe4bc0, 0x2d, 0xc422096ca0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequestHeader(0xc420e41540, 0x1d72340, 0xc421ac5e00, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420200eb0, 0x8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequest(0xc420e41540, 0x1d72340, 0xc421ac5e00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc420e41540, 0x1d72340, 0xc421ac5e00, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc42096d440, 0xc420e41540) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 262 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4223d0000, 0xc421279d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4223d0000, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420204040, 0xc420bd6000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420204040, 0xc420bd6000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 166 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420072160, 0xc420063d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420072160, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42000ed20, 0xc420e10960) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42000ed20, 0xc420e10960) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 159 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30880, 0x72, 0xc4212f29a8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc4200f1e98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc4200f1e98, 0xc420a80000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc4200f1e80, 0xc420a80000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc4200f1e80, 0xc420a80000, 0x1000, 0x1000, 0x458600, 0xc420ae2c00, 0x4) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420216e88, 0xc420a80000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*persistConn).Read(0xc420b96900, 0xc420a80000, 0x1000, 0x1000, 0xc4212f2b98, 0x405675, 0xc420df1aa0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc420a7e1e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Peek(0xc420a7e1e0, 0x1, 0x0, 0x0, 0x1, 0xc420cdf260, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*persistConn).readLoop(0xc420b96900) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by net/http.(*Transport).dialConn Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 160 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*persistConn).writeLoop(0xc420b96900) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by net/http.(*Transport).dialConn Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 174 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200722c0, 0xc4212f7d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200722c0, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42000f0d0, 0xc420e11270) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42000f0d0, 0xc420e11270) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 168 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420072160) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 169 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420072160) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 269 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92ee38, 0x72, 0xc42147ace8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc422376318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc422376318, 0xc422385000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc422376300, 0xc422385000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc422376300, 0xc422385000, 0x1000, 0x1000, 0xc4201afdd8, 0x1, 0xc4201afe08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420503b48, 0xc422385000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc4229ee060, 0xc422954a50, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc4229ee060, 0xc422954a50, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc4229ee060, 0xc422954a50, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421c2c580, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421c2c580) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 212 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30200, 0x72, 0xc420e97ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc421390018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc421390018, 0xc420bfe000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc421390000, 0xc420bfe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc421390000, 0xc420bfe000, 0x1000, 0x1000, 0xc420e97dd8, 0x0, 0xc420e97e08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420204a80, 0xc420bfe000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc420c32840, 0xc420565170, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc420c32840, 0xc420565170, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc420c32840, 0xc420565170, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420445290, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420445290) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 181 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba307b0, 0x72, 0xc4212d8ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420af1198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420af1198, 0xc420543000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc420af1180, 0xc420543000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc420af1180, 0xc420543000, 0x1000, 0x1000, 0xc4212d8dd8, 0x0, 0xc4212d8e08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420217168, 0xc420543000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc420c32fc0, 0xc420533cc0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc420c32fc0, 0xc420533cc0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc420c32fc0, 0xc420533cc0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42023a9a0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42023a9a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 182 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42023a9a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 183 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42023a9a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 184 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4210962a0, 0xc4212fb700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420c938c0, 0x1d718c0, 0xc4210962a0, 0x0, 0xc421096200) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420422000, 0xc4212fb950, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420b92d80, 0xc4213ce000, 0xc420c93770, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc420086900, 0xc420217470, 0x13, 0x1bda9a0, 0x4, 0xc4212fbd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc420086900, 0xc420217470, 0x13, 0xc4212fbd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc420b53c80, 0xc420e68280, 0xc42102a618, 0x0, 0xc420b91a80, 0xc42061da40, 0x19758e0, 0xc4213ce000, 0x16, 0x169e020, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc420e68280, 0x1d72340, 0xc420b53d00, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cc300, 0xc420e68280) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 175 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba306e0, 0x72, 0xc420e92ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420bc4318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420bc4318, 0xc42146a000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc420bc4300, 0xc42146a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc420bc4300, 0xc42146a000, 0x1000, 0x1000, 0xc420e92dd8, 0x0, 0xc420e92e08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc42000f0d0, 0xc42146a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc420aff200, 0xc42052efd0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc420aff200, 0xc42052efd0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc420aff200, 0xc42052efd0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200722c0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200722c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 176 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200722c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 177 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200722c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 432 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42096d800, 0xc422320000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc422bdd500) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc422bdd500, 0xc420042500, 0xc4218c49d8, 0xc4218c49e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421621500, 0xc4231d9448) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421177320) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422bdd560, 0x16e4ba0, 0xc4216219e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422bdd560, 0x16e4ba0, 0xc4216219e0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4218c5340, 0x16e4ba0, 0xc4216219e0, 0x1, 0xc4216219e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc4218c5340, 0xc4216219e0, 0x1974aa0, 0xc422bdd3e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc4218c5340, 0x1be6a7c, 0xc, 0x1974aa0, 0xc422bdd3e0, 0x1695860, 0xc4211772a0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc420228360, 0xc420213411, 0x6, 0x1d62b00, 0xc420866870, 0x1, 0x1be6a7c, 0xc, 0x1974aa0, 0xc422bdd3e0, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc420422000, 0xc4209228c0, 0x1be6a7c, 0xc, 0x1974aa0, 0xc422bdd3e0, 0x1695860, 0xc4211772a0, 0x1947b20, 0xc4230388c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420422000, 0x1be6a7c, 0xc, 0x1d76c80, 0xc422bdd3e0, 0x1974aa0, 0xc422bdd3e0, 0x1695860, 0xc4211772a0, 0xc423038968, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Status).Peers(0xc420216220, 0xc422bdd3e0, 0xc4211772a0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/status_endpoint.go:64 +0xd1 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc4200861e0, 0xc42000f7b0, 0x13, 0x1bda9a0, 0x4, 0xc423038d48, 0x3, 0x3, 0x1709fe0, 0xc4211772a0, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc4200861e0, 0xc42000f7b0, 0x13, 0xc423038d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc421f9bb40, 0xc4213deb40, 0xc420666a00, 0x0, 0xc421481d80, 0xc421177260, 0x1974aa0, 0xc422bdd3e0, 0x16, 0x1695860, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc4213deb40, 0x1d72340, 0xc4218c51c0, 0x0, 0xc42096d740) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc42096d740, 0xc4213deb40) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 185 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200d6630, 0xc420e98d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200d6630, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc4205025e0, 0xc420e41a90) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc4205025e0, 0xc420e41a90) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 186 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30610, 0x72, 0xc42147cce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420e13d18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420e13d18, 0xc420d09000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc420e13d00, 0xc420d09000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc420e13d00, 0xc420d09000, 0x1000, 0x1000, 0xc42147cdd8, 0x0, 0xc42147ce08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc4205025e0, 0xc420d09000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc420de93e0, 0xc42052f000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc420de93e0, 0xc42052f000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc420de93e0, 0xc42052f000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200d6630, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200d6630) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 187 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200d6630) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 188 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200d6630) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 223 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30060, 0x72, 0xc4212d7ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc421118018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc421118018, 0xc420e71000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc421118000, 0xc420e71000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc421118000, 0xc420e71000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420204040, 0xc420e71000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc420e6ec00, 0xc422162640, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc420e6ec00, 0xc422162640, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc420e6ec00, 0xc422162640, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4223d0000, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4223d0000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 304 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421c2c6e0, 0xc4212d6d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421c2c6e0, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420217920, 0xc420e68910) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420217920, 0xc420e68910) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 206 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420cae580) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 444 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc421b203c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 214 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420445290) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 213 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420445290) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 227 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200d68f0, 0xc4212d5d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200d68f0, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420216068, 0xc42117a1e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420216068, 0xc42117a1e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 228 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba302d0, 0x72, 0xc420187ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420b40118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420b40118, 0xc420d57000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc420b40100, 0xc420d57000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc420b40100, 0xc420d57000, 0x1000, 0x1000, 0xc420187dd8, 0x0, 0xc420187e08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420216068, 0xc420d57000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc4201f6c60, 0xc42099c460, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc4201f6c60, 0xc42099c460, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc4201f6c60, 0xc42099c460, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200d68f0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200d68f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 229 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200d68f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 230 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200d68f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 439 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4200c7860) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 232 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42023a160, 0xc420e95d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42023a160, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc4205025d0, 0xc4203ad6d0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc4205025d0, 0xc4203ad6d0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 233 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30540, 0x72, 0xc421276ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc421064018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc421064018, 0xc421312000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc421064000, 0xc421312000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc421064000, 0xc421312000, 0x1000, 0x1000, 0xc421276dd8, 0x0, 0xc421276e08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc4205025d0, 0xc421312000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc420946480, 0xc42093cbd0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc420946480, 0xc42093cbd0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc420946480, 0xc42093cbd0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42023a160, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42023a160) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 234 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42023a160) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 235 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42023a160) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 270 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421c2c580) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 243 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420445290, 0xc421274d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420445290, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420204a80, 0xc420531d10) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420204a80, 0xc420531d10) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 224 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4223d0000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 225 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4223d0000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 275 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420445340, 0xc421277d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420445340, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420502c80, 0xc422a66cd0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420502c80, 0xc422a66cd0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 276 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba303a0, 0x72, 0xc421479ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc421ec8018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc421ec8018, 0xc421332000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc421ec8000, 0xc421332000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc421ec8000, 0xc421332000, 0x1000, 0x1000, 0x92, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420502c80, 0xc421332000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc422a63f20, 0xc4227830d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc422a63f20, 0xc4227830d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc422a63f20, 0xc4227830d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420445340, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420445340) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 277 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420445340) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 278 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420445340) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 279 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4208cc180, 0xc422370000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc422090e40) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc422090e40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42237e9c0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42216ac80) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422090ea0, 0x16e4b60, 0xc42216ace0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422090ea0, 0x16e4b60, 0xc42216ace0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4223a7340, 0x16e4b60, 0xc42216ace0, 0xc42216ace0, 0xc422a66d98) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4223a7340, 0xc42216ace0, 0x2d, 0xc422116740) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequestHeader(0xc422a66d70, 0x1d72340, 0xc4223a7340, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420200eb0, 0x8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequest(0xc422a66d70, 0x1d72340, 0xc4223a7340, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc422a66d70, 0x1d72340, 0xc4223a7340, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cc180, 0xc422a66d70) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 280 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42102c300, 0xc42227e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc4223180c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc4223180c0, 0xc420044a00, 0xc421d1e258, 0xc421d1e260) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421aa4060, 0xc420f03448) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42060d660) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422318120, 0x16e4ba0, 0xc421aa4510) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422318120, 0x16e4ba0, 0xc421aa4510, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4220880c0, 0x16e4ba0, 0xc421aa4510, 0x1, 0xc421aa4510) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc4220880c0, 0xc421aa4510, 0x1974aa0, 0xc42050ff80) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc4220880c0, 0x1be6a7c, 0xc, 0x1974aa0, 0xc42050ff80, 0x1695860, 0xc42060d5e0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc420228360, 0xc420213411, 0x6, 0x1d62b00, 0xc420866870, 0x1, 0x1be6a7c, 0xc, 0x1974aa0, 0xc42050ff80, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc420422000, 0xc4209228c0, 0x1be6a7c, 0xc, 0x1974aa0, 0xc42050ff80, 0x1695860, 0xc42060d5e0, 0x1947b20, 0xc420f038c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420422000, 0x1be6a7c, 0xc, 0x1d76c80, 0xc42050ff80, 0x1974aa0, 0xc42050ff80, 0x1695860, 0xc42060d5e0, 0xc420f03968, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Status).Peers(0xc420216220, 0xc42050ff80, 0xc42060d5e0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/status_endpoint.go:64 +0xd1 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc4200861e0, 0xc420502ee8, 0x13, 0x1bda9a0, 0x4, 0xc420f03d48, 0x3, 0x3, 0x1709fe0, 0xc42060d5e0, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc4200861e0, 0xc420502ee8, 0x13, 0xc420f03d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc422a61700, 0xc422a66d70, 0xc4202070e8, 0x0, 0xc4222a8180, 0xc422a5f520, 0x1974aa0, 0xc42050ff80, 0x16, 0x1695860, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc422a66d70, 0x1d72340, 0xc422a618c0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cc240, 0xc422a66d70) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 336 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421c2c840) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 339 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92eaf8, 0x72, 0xc4212e0658) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc4227f1c98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc4227f1c98, 0xc4205d3900, 0x1, 0x1) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc4227f1c80, 0xc4205d39f1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc4227f1c80, 0xc4205d39f1, 0x1, 0x1, 0x74696157a9007469, 0x1af6c69746e55, 0x1eb90f0ed30e0000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420204eb8, 0xc4205d39f1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*connReader).backgroundRead(0xc4205d39e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 347 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421c2c6e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 204 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420cae580, 0xc420183d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420cae580, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42114cc48, 0xc422bfed70) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42114cc48, 0xc422bfed70) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 207 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420cae580) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 208 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42228a3c0, 0xc4230a7700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42127d470, 0x1d718c0, 0xc42228a3c0, 0x0, 0xc42228a300) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420422000, 0xc4230a7950, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc422645c20, 0xc422aa9800, 0xc42127d260, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc420086900, 0xc42114cf50, 0x13, 0x1bda9a0, 0x4, 0xc4230a7d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc420086900, 0xc42114cf50, 0x13, 0xc4230a7d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc420aed100, 0xc422bfee10, 0xc42247c8c8, 0x0, 0xc422aa8d00, 0xc42129ff80, 0x19758e0, 0xc422aa9800, 0x16, 0x169e020, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc422bfee10, 0x1d72340, 0xc420aed140, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cd200, 0xc422bfee10) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 209 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421c2c580, 0xc4212f8d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421c2c580, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420503b48, 0xc4214c1630) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420503b48, 0xc4214c1630) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 271 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421c2c580) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 272 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4208cd2c0, 0xc421efb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc4224886c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc4224886c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4213a6270, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420e338c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422488720, 0x16e4b60, 0xc420e33920) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422488720, 0x16e4b60, 0xc420e33920, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4205d9700, 0x16e4b60, 0xc420e33920, 0xc420e33920, 0xc42316a488) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4205d9700, 0xc420e33920, 0x2d, 0xc421ff4fc0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequestHeader(0xc42316a460, 0x1d72340, 0xc4205d9700, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420200eb0, 0x8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequest(0xc42316a460, 0x1d72340, 0xc4205d9700, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc42316a460, 0x1d72340, 0xc4205d9700, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cd2c0, 0xc42316a460) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 348 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421c2c6e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 292 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4208821e0, 0xc42312b2c8, 0x20, 0x20, 0x5, 0x1bd9a98) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4208213e0, 0x1d718c0, 0xc4208821e0, 0x0, 0xc420882100) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420422000, 0xc42312b500, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Node).List(0xc4200866c0, 0xc420c33380, 0xc420820420, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:1198 +0x27c Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc4200869c0, 0xc420216e00, 0x13, 0x1bda9a0, 0x4, 0xc4239ff8f8, 0x3, 0x3, 0x192ba60, 0x19, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc4200869c0, 0xc420216e00, 0x13, 0xc4231d38f8, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc420210a80, 0xc42020a550, 0xc4225d08c8, 0x0, 0xc4200f0f00, 0xc420195b80, 0x1975760, 0xc420c33380, 0x16, 0x169e120, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc42020a550, 0x1d71b80, 0xc42153a000, 0xc4204a8400, 0xa) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420422000, 0x1be205d, 0x9, 0x1975760, 0xc420c33320, 0x169e120, 0xc4208203f0, 0xc420c33301, 0xc4208203f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc42033a070, 0x1be205d, 0x9, 0x1975760, 0xc420c33320, 0x169e120, 0xc4208203f0, 0xc421a7aa80, 0x1e) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest(0xc420b52240, 0x1d6df80, 0xc420c332c0, 0xc420e30600, 0xc421a7aab0, 0xc4231d3b58, 0x38, 0xc4231d3b50) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/node_endpoint.go:24 +0x1b3 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest-fm(0x1d6df80, 0xc420c332c0, 0xc420e30600, 0xc420e30600, 0xc4207f9760, 0x1e, 0xbed7c80447ed6155) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:145 +0x48 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc420c332c0, 0xc420e30600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.HandlerFunc.ServeHTTP(0xc420650ac0, 0x1d6df80, 0xc420c332c0, 0xc420e30600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*ServeMux).ServeHTTP(0xc420b542a0, 0x1d6df80, 0xc420c332c0, 0xc420e30600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc420ce5880, 0xc420e30600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.HandlerFunc.ServeHTTP(0xc420e0f6b0, 0x1d6fec0, 0xc420ce5880, 0xc420e30600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.serverHandler.ServeHTTP(0xc4211892b0, 0x1d6fec0, 0xc420ce5880, 0xc420e30600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*conn).serve(0xc423031540, 0x1d71840, 0xc421083700) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by net/http.(*Server).Serve Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 256 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92ed68, 0x72, 0xc4231d0ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc421391118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc421391118, 0xc423229000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc421391100, 0xc423229000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc421391100, 0xc423229000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc42000f9b0, 0xc423229000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc4226bfd40, 0xc423248b70, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc4226bfd40, 0xc423248b70, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc4226bfd40, 0xc423248b70, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200d6b00, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200d6b00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 257 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200d6b00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 306 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200d6b00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 307 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420cc5620, 0xc4231db700, 0x20, 0x20, 0x169d4e0, 0xc4218d1680) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42320d9e0, 0x1d718c0, 0xc420cc5620, 0x0, 0xc420cc5600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420422000, 0xc4231db950, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc422891e00, 0xc420b9a780, 0xc42320d8f0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc420086900, 0xc42000fce0, 0x13, 0x1bda9a0, 0x4, 0xc4231dbd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc420086900, 0xc42000fce0, 0x13, 0xc4231dbd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc423250740, 0xc4232dea50, 0xc422ccf708, 0x0, 0xc420bc6480, 0xc4232aee40, 0x19758e0, 0xc420b9a780, 0x16, 0x169e020, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc4232dea50, 0x1d72340, 0xc423250800, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc420074240, 0xc4232dea50) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 322 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4204454a0, 0xc4231ccd10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4204454a0, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc4202170e0, 0xc4220d8b90) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc4202170e0, 0xc4220d8b90) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 323 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92ec98, 0x72, 0xc422edace8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420eb5a98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420eb5a98, 0xc422def000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc420eb5a80, 0xc422def000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc420eb5a80, 0xc422def000, 0x1000, 0x1000, 0xc4201af5d8, 0x1, 0xc4201af608) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc4202170e0, 0xc422def000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc422644300, 0xc422522dd0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc422644300, 0xc422522dd0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc422644300, 0xc422522dd0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4204454a0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4204454a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 324 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4204454a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 325 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4204454a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 326 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4208cc480, 0xc4220da000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc420cc4c00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc420cc4c00, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42320c210, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422a404e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420cc4c60, 0x16e4b60, 0xc422a40540) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420cc4c60, 0x16e4b60, 0xc422a40540, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422e62300, 0x16e4b60, 0xc422a40540, 0xc422a40540, 0xc4220d8c58) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422e62300, 0xc422a40540, 0x2d, 0xc42225e280) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequestHeader(0xc4220d8c30, 0x1d72340, 0xc422e62300, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420200eb0, 0x8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequest(0xc4220d8c30, 0x1d72340, 0xc422e62300, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc4220d8c30, 0x1d72340, 0xc422e62300, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cc480, 0xc4220d8c30) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 443 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420256480) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 442 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420256480) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 295 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420afe9c0, 0xc4239fb208, 0x20, 0x20, 0xc422436980, 0xc420287620) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420287950, 0x1d718c0, 0xc420afe9c0, 0x0, 0xc420afe900) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420422000, 0xc4239fb440, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Job).List(0xc4202161f0, 0xc420afe8a0, 0xc420287620, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/job_endpoint.go:925 +0x2ea Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc420229620, 0xc420216778, 0x13, 0x1bda9a0, 0x4, 0xc4239fb898, 0x3, 0x3, 0x192b7e0, 0x19, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc420229620, 0xc420216778, 0x13, 0xc4231cd898, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc4202102c0, 0xc42020a550, 0xc421eb7600, 0x0, 0xc42021f780, 0xc4205282a0, 0x1974e60, 0xc420afe8a0, 0x16, 0x169dd20, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc42020a550, 0x1d71b80, 0xc421503400, 0xc4204a8400, 0xa) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420422000, 0x1be08be, 0x8, 0x1974e60, 0xc420afe840, 0x169dd20, 0xc420287590, 0xc420afe801, 0xc420287590) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc42033a070, 0x1be08be, 0x8, 0x1974e60, 0xc420afe840, 0x169dd20, 0xc420287590, 0x0, 0xc4204a8400) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).jobListRequest(0xc420b52240, 0x1d6df80, 0xc420afe7e0, 0xc420df2700, 0x44774f, 0xc421fe0f00, 0xc422053ce0, 0x1d) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:33 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest(0xc420b52240, 0x1d6df80, 0xc420afe7e0, 0xc420df2700, 0xc422053d10, 0xc4231cdb58, 0x38, 0xc4231cdb50) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:18 +0x1a6 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest-fm(0x1d6df80, 0xc420afe7e0, 0xc420df2700, 0xc420df2700, 0xc420825980, 0x1d, 0xbed7c80448043bd3) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:141 +0x48 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc420afe7e0, 0xc420df2700) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.HandlerFunc.ServeHTTP(0xc420650a60, 0x1d6df80, 0xc420afe7e0, 0xc420df2700) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*ServeMux).ServeHTTP(0xc420b542a0, 0x1d6df80, 0xc420afe7e0, 0xc420df2700) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc420ca5ce0, 0xc420df2700) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.HandlerFunc.ServeHTTP(0xc420e0f6b0, 0x1d6fec0, 0xc420ca5ce0, 0xc420df2700) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.serverHandler.ServeHTTP(0xc4211892b0, 0x1d6fec0, 0xc420ca5ce0, 0xc420df2700) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*conn).serve(0xc423031cc0, 0x1d71840, 0xc421861540) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by net/http.(*Server).Serve Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 360 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4225ac000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 327 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92ebc8, 0x72, 0xc4212cbe58) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc4227f1818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc4227f1818, 0xc420820000, 0x1, 0x1) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc4227f1800, 0xc420820041, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc4227f1800, 0xc420820041, 0x1, 0x1, 0xc4202ec540, 0xf9896a, 0x1) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420204d48, 0xc420820041, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/http.(*connReader).backgroundRead(0xc420820030) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 361 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4225ac000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 373 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30130, 0x72, 0xc4212f9ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc4219bdf98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc4219bdf98, 0xc421ab5000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc4219bdf80, 0xc421ab5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc4219bdf80, 0xc421ab5000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420205850, 0xc421ab5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc4220e6840, 0xc420c10850, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc4220e6840, 0xc420c10850, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc4220e6840, 0xc420c10850, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421c2c8f0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421c2c8f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 372 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421c2c8f0, 0xc4231cdd10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421c2c8f0, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420205850, 0xc420e411d0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420205850, 0xc420e411d0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 359 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92efd8, 0x72, 0xc422ed6ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc422086018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc422086018, 0xc422534000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc422086000, 0xc422534000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc422086000, 0xc422534000, 0x1000, 0x1000, 0xc422ed6dd8, 0x0, 0xc422ed6e08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc42114c5e8, 0xc422534000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc421824780, 0xc42093d850, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc421824780, 0xc42093d850, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc421824780, 0xc42093d850, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4225ac000, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4225ac000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 333 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421c2c840, 0xc4231ced10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421c2c840, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420205430, 0xc420e40eb0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420205430, 0xc420e40eb0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 438 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4200c7860) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 315 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92ef08, 0x72, 0xc4231d2ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc422376998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc422376998, 0xc421358000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc422376980, 0xc421358000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc422376980, 0xc421358000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420217940, 0xc421358000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc420b92a20, 0xc420ff67e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc420b92a20, 0xc420ff67e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc420b92a20, 0xc420ff67e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420073ce0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420073ce0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 316 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420073ce0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 317 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420073ce0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 318 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4208cd680, 0xc421359000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc421b497a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc421b497a0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42113d950, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206ae6e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421b49980, 0x16e4b60, 0xc4206ae760) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421b49980, 0x16e4b60, 0xc4206ae760, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421d58ec0, 0x16e4b60, 0xc4206ae760, 0xc4206ae760, 0xc42019c528) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421d58ec0, 0xc4206ae760, 0x2d, 0xc4214e2240) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequestHeader(0xc42019c500, 0x1d72340, 0xc421d58ec0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420200eb0, 0x8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequest(0xc42019c500, 0x1d72340, 0xc421d58ec0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc42019c500, 0x1d72340, 0xc421d58ec0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cd680, 0xc42019c500) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 353 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420caf3f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 374 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421c2c8f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 375 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421c2c8f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 380 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421c2c9a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 410 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420caf760) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 363 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4225ac0b0, 0xc422eddd10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4225ac0b0, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42114c940, 0xc420e68eb0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42114c940, 0xc420e68eb0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 364 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92e958, 0x72, 0xc42303cce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc4225bce18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc4225bce18, 0xc422853000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc4225bce00, 0xc422853000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc4225bce00, 0xc422853000, 0x1000, 0x1000, 0xc42303cdd8, 0x1, 0xc42303ce08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc42114c940, 0xc422853000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc4227615c0, 0xc420c108d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc4227615c0, 0xc420c108d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc4227615c0, 0xc420c108d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4225ac0b0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4225ac0b0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 365 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4225ac0b0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 366 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4225ac0b0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 379 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421c2c9a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 377 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421c2c9a0, 0xc422ed7d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421c2c9a0, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420205ba8, 0xc420e41450) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420205ba8, 0xc420e41450) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 378 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92e888, 0x72, 0xc4204d8ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc42170ce18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc42170ce18, 0xc4226e2000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc42170ce00, 0xc4226e2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc42170ce00, 0xc4226e2000, 0x1000, 0x1000, 0xc4204d8dd8, 0x1, 0xc4204d8e08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420205ba8, 0xc4226e2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc4223f5380, 0xc4209a58d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc4223f5380, 0xc4209a58d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc4223f5380, 0xc4209a58d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421c2c9a0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421c2c9a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 351 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92e7b8, 0x72, 0xc42303ece8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc4227c4218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc4227c4218, 0xc4216db000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc4227c4200, 0xc4216db000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc4227c4200, 0xc4216db000, 0x1000, 0x1000, 0xc42303edd8, 0x0, 0xc42303ee08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc420217960, 0xc4216db000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc422090960, 0xc4209a5980, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc422090960, 0xc4209a5980, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc422090960, 0xc4209a5980, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420caf3f0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420caf3f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 352 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420caf3f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 383 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420caf3f0, 0xc4231d1d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420caf3f0, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420217960, 0xc421434280) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc420217960, 0xc421434280) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 384 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420caf600, 0xc42303dd10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420caf600, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42114d020, 0xc42117a0a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42114d020, 0xc42117a0a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 421 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422ea0180, 0xc4239fd700, 0x20, 0x20, 0x169d4e0, 0xc4218d1680) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4216acf60, 0x1d718c0, 0xc422ea0180, 0x0, 0xc422ea0100) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420422000, 0xc4239fd950, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4223f4a20, 0xc422626800, 0xc4216acb70, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc420086900, 0xc420205b58, 0x13, 0x1bda9a0, 0x4, 0xc4239fdd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc420086900, 0xc420205b58, 0x13, 0xc4239fdd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc422108100, 0xc420e41270, 0xc42046ae38, 0x0, 0xc42170c880, 0xc4211cb240, 0x19758e0, 0xc422626800, 0x16, 0x169e020, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc420e41270, 0x1d72340, 0xc421f1de80, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cdb00, 0xc420e41270) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 407 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4208cdd40, 0xc421edc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc42293d5c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc42293d5c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42186a090, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4205e6060) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42293d620, 0x16e4b60, 0xc4205e6100) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42293d620, 0x16e4b60, 0xc4205e6100, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42217d080, 0x16e4b60, 0xc4205e6100, 0xc4205e6100, 0xc420e68f78) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42217d080, 0xc4205e6100, 0x2d, 0xc421eb4240) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequestHeader(0xc420e68f50, 0x1d72340, 0xc42217d080, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420200eb0, 0x8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequest(0xc420e68f50, 0x1d72340, 0xc42217d080, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc420e68f50, 0x1d72340, 0xc42217d080, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cdd40, 0xc420e68f50) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 422 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92e548, 0x72, 0xc422ed8ce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc420dac418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc420dac418, 0xc421ea3000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc420dac400, 0xc421ea3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc420dac400, 0xc421ea3000, 0x1000, 0x1000, 0xac, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc42114d020, 0xc421ea3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc421c6ec00, 0xc42046be80, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc421c6ec00, 0xc42046be80, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc421c6ec00, 0xc42046be80, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420caf600, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420caf600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 423 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420caf600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 424 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420caf600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 425 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422008060, 0xc421301700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4211f43c0, 0x1d718c0, 0xc422008060, 0x0, 0xc422008000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420422000, 0xc421301950, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4208def00, 0xc422660000, 0xc4211f4120, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc420086900, 0xc420217f58, 0x13, 0x1bda9a0, 0x4, 0xc420188d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc420086900, 0xc420217f58, 0x13, 0xc420188d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc421dd95c0, 0xc420e41f90, 0xc4202dce38, 0x0, 0xc4227c5c80, 0xc4211cb760, 0x19758e0, 0xc422660000, 0x16, 0x169e020, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc420e41f90, 0x1d72340, 0xc421dd9600, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc420355d40, 0xc420e41f90) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 426 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4208cdc80, 0xc422664000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc4208df980) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc4208df980, 0x101000000000016, 0x0, 0x2d) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4216dc720, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4211cb8c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4208df9e0, 0x16e4b60, 0xc4211cb920) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4208df9e0, 0x16e4b60, 0xc4211cb920, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421dd9c00, 0x16e4b60, 0xc4211cb920, 0xc4211cb920, 0xc420e41fb8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421dd9c00, 0xc4211cb920, 0x2d, 0xc421dac6c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequestHeader(0xc420e41f90, 0x1d72340, 0xc421dd9c00, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420200eb0, 0x8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).readRequest(0xc420e41f90, 0x1d72340, 0xc421dd9c00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc420e41f90, 0x1d72340, 0xc421dd9c00, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cdc80, 0xc420e41f90) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 400 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92e6e8, 0x72, 0xc422a7d4e8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc422660718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc422660718, 0xc421ca2000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc422660700, 0xc421ca2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc422660700, 0xc421ca2000, 0x1000, 0x1000, 0xc422a7d5d8, 0x1, 0xc422a7d608) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc4226102e0, 0xc421ca2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc422ea14a0, 0xc420667150, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc422ea14a0, 0xc420667150, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc422ea14a0, 0xc420667150, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4213eab00, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4213eab00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 408 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42293dbc0, 0xc4230ad700, 0x20, 0x20, 0x169d4e0, 0xc4218d1680) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42186bb90, 0x1d718c0, 0xc42293dbc0, 0x0, 0xc42293db00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420422000, 0xc4230ad950, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421886900, 0xc42266c480, 0xc42186b860, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc420086900, 0xc420205370, 0x13, 0x1bda9a0, 0x4, 0xc421272d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc420086900, 0xc420205370, 0x13, 0xc421272d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc4216a40c0, 0xc421ad9d60, 0xc420564688, 0x0, 0xc421e13a00, 0xc4205e6360, 0x19758e0, 0xc42266c480, 0x16, 0x169e020, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc421ad9d60, 0x1d72340, 0xc42217d5c0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc4208cdec0, 0xc421ad9d60) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 411 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420caf760) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 466 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421c2cb00, 0xc423039d10, 0x30, 0xf9866a) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421c2cb00, 0x1c61988, 0xc420422000, 0x1d71840, 0xc420b52180) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42114d050, 0xc4212a9450) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d320, 0xc42114d050, 0xc4212a9450) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 414 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4203bc000, 0xc42102c540, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4203bc000, 0xc42102c540) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203bc000, 0xc42060d900) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 437 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4203bc000, 0xc42102c480, 0xc42222e5a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203bc000, 0xc4205e7aa0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 401 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4213eab00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 450 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4213eab00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 467 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfba30470, 0x72, 0xc4231cfce8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc422660918, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc422660918, 0xc421d86000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc422660900, 0xc421d86000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc422660900, 0xc421d86000, 0x1000, 0x1000, 0xc4231cfdd8, 0x1, 0xc4231cfe08) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc42114d050, 0xc421d86000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).Read(0xc422bdd9e0, 0xc420e256c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadAtLeast(0x1d59aa0, 0xc422bdd9e0, 0xc420e256c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: io.ReadFull(0x1d59aa0, 0xc422bdd9e0, 0xc420e256c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421c2cb00, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421c2cb00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 468 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421c2cb00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 469 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421c2cb00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 470 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc422214180, 0xc420aa9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc420b1e060) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc420b1e060, 0xc420042500, 0xc420e2c798, 0xc420e2c7a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421af0cc0, 0xc4212ff448) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421177d80) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420b1e0c0, 0x16e4ba0, 0xc421af11d0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420b1e0c0, 0x16e4ba0, 0xc421af11d0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422397280, 0x16e4ba0, 0xc421af11d0, 0x1, 0xc421af11d0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc422397280, 0xc421af11d0, 0x1974aa0, 0xc4214c9e60) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc422397280, 0x1be6a7c, 0xc, 0x1974aa0, 0xc4214c9e60, 0x1695860, 0xc421177ca0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc420228360, 0xc420213411, 0x6, 0x1d62b00, 0xc420866870, 0x1, 0x1be6a7c, 0xc, 0x1974aa0, 0xc4214c9e60, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc420422000, 0xc4209228c0, 0x1be6a7c, 0xc, 0x1974aa0, 0xc4214c9e60, 0x1695860, 0xc421177ca0, 0x1947b20, 0xc422ed98c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420422000, 0x1be6a7c, 0xc, 0x1d76c80, 0xc4214c9e60, 0x1974aa0, 0xc4214c9e60, 0x1695860, 0xc421177ca0, 0xc422ed9968, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Status).Peers(0xc420216220, 0xc4214c9e60, 0xc421177ca0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/status_endpoint.go:64 +0xd1 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.call(0xc4200861e0, 0xc42114d2a0, 0x13, 0x1bda9a0, 0x4, 0xc422ed9d48, 0x3, 0x3, 0x1709fe0, 0xc421177ca0, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: reflect.Value.Call(0xc4200861e0, 0xc42114d2a0, 0x13, 0xc422ed9d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*service).call(0xc422396740, 0xc4212a94f0, 0xc420864410, 0x0, 0xc422627a80, 0xc421177c60, 0x1974aa0, 0xc4214c9e60, 0x16, 0x1695860, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net/rpc.(*Server).ServeRequest(0xc4212a94f0, 0x1d72340, 0xc422396e40, 0x0, 0xc42096d8c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420422000, 0x1d71840, 0xc420b52180, 0x1d7d0e0, 0xc42096d8c0, 0xc4212a94f0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 413 [IO wait]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.runtime_pollWait(0x7f4bfb92e478, 0x72, 0xc4230e8858) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).wait(0xc421746098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*pollDesc).waitRead(0xc421746098, 0xc42116e000, 0x1000, 0x1000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: internal/poll.(*FD).Read(0xc421746080, 0xc42116e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*netFD).Read(0xc421746080, 0xc42116e000, 0x1000, 0x1000, 0xc420040000, 0xc4232508d8, 0xc4232508e0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: net.(*conn).Read(0xc42114d3c0, 0xc42116e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).fill(0xc420b1e420) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: bufio.(*Reader).ReadByte(0xc420b1e420, 0x72, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421af1dd0, 0xc421746098) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4218a42a0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420b1e480, 0x16e76e0, 0xc4204ed750) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420b1e480, 0x16e76e0, 0xc4204ed750, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.decodeResponse(0xc422397780, 0x182ba60, 0xc422170900, 0xc4222ef8c0, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:608 +0x5e Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).genericRPC(0xc420072210, 0xc420359530, 0x24, 0xc420291fe0, 0x11, 0x14019996222ef800, 0x182ba00, 0xc4222ef8c0, 0x182ba60, 0xc422170900, ...) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:406 +0xf9 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).AppendEntries(0xc420072210, 0xc420359530, 0x24, 0xc420291fe0, 0x11, 0xc4222ef8c0, 0xc422170900, 0x1, 0x3) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:379 +0x97 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4203bc000, 0xc42102c480, 0x1863ed, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:186 +0x189 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4203bc000, 0xc42102c480) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:132 +0x3f2 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203bc000, 0xc42060d8c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 474 [runnable]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldBlock(0xc4212542c0, 0x1a1c000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6607 +0x2a4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc420422000, 0xc4203af801, 0xc4203af800) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:300 +0x152 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc420422000, 0xc421ebcc60, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc420422000, 0xc421ebcc60) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420686160, 0xc420422000, 0xc421ebcc60) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 475 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4203bc000, 0xc42102c540, 0xc4220a0600) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203bc000, 0xc4218a45c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 415 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc422088b00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 416 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4203bc000, 0xc42102c540, 0x1d6eb80, 0xc422088b00, 0xc4224e2e40, 0xc4224e3740) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203bc000, 0xc422088b40) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 445 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4224368c0, 0xc42303fce8, 0x20, 0x20, 0x18, 0xc4216e3140) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4218f94d0, 0x1d71840, 0xc4224368c0, 0xc4216e3140, 0x184987) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203af800, 0xc422b55ed0, 0x184987, 0x1d71840, 0xc4224368c0, 0xc420903060, 0xc421b20330, 0xc420815500, 0x24, 0x2bba400) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420903000, 0x1d71840, 0xc4224368c0, 0x184987, 0xc422364a00, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420903000, 0x1d71840, 0xc4224368c0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 446 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422436d00, 0xc4212eb770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4218f9650, 0x1d71840, 0xc422436d00, 0x0, 0x1849b5) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203af800, 0xc4212eb960, 0x1849b5, 0x1d71840, 0xc422436d00, 0x552bde, 0xc4212eb950, 0xc4212eb970, 0xf33ba5, 0xc4212c1cc8) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc4212c1c70, 0x1d71840, 0xc422436d00, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc4212c1c70) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 447 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422436cc0, 0xc420f09a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421b0af60, 0x1d71840, 0xc422436cc0, 0xc421b0b230, 0x1833a4) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203af800, 0xc420f09c68, 0x1833a4, 0x1d71840, 0xc422436cc0, 0x552bde, 0xc42020a190, 0x2, 0xc42169c980, 0x40) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422436f80, 0x1833a4, 0x3b, 0xc4230e6de8, 0x1, 0x1) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422436f80) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 448 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422437000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 449 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420b665a0, 0x1d71840, 0xc422436cc0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 417 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc421b3efc0, 0xc4224e3860) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 482 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc421b3efc0, 0xc4224e3a40) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 451 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc42262c000) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: goroutine 483 [select]: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422364e40, 0xc42303ad28, 0x20, 0x20, 0xc4204a8400, 0x1bd9a98) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421b3f4a0, 0x1d71840, 0xc422364e40, 0xc4218d0480, 0x1849b5) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4203af800, 0xc420f07f20, 0x1849b5, 0x1d71840, 0xc422364e40, 0x0, 0x0, 0x0, 0x0, 0xc4218e5c20) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc42262c000, 0x1849b5, 0xc4212dbfb8, 0xc4212dbee8, 0xc422319080, 0xf959ca, 0x1, 0x0) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc42262c000, 0x1849b5, 0xc422319620) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26008]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:29:12 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:29:12 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:29:12 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: ==> Starting Nomad agent... Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: ==> Nomad agent configuration: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: Client: false Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: Log Level: INFO Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: Region: amrv01 (DC: test) Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: Server: true Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: Version: 0.8.4 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:12 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:12 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:12 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:12 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:12.971726 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:12 [INFO] serf: Attempting re-join to previously known node: ip-172-27-0-204.aav01: 172.27.0.204:4648 Aug 23 15:29:12 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:12.971865 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.012384 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.012402 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326035 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326058 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326070 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326314 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326331 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326344 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326355 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326367 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326379 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326390 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326401 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326504 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326521 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326548 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [WARN] memberlist: Refuting an alive message Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326580 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326772 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326788 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13.326815 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:29:13 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:13 [INFO] serf: Re-joined to previously known node: ip-172-27-0-204.aav01: 172.27.0.204:4648 Aug 23 15:29:14 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:14 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:29:14 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:14 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 63 Aug 23 15:29:14 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:14 [INFO] raft: Election won. Tally: 2 Aug 23 15:29:14 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:14 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:29:14 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:14 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:29:14 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:14 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:29:14 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:14.685279 [INFO] nomad: cluster leadership acquired Aug 23 15:29:14 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:14 [WARN] raft: AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} rejected, sending older logs (next: 1598446) Aug 23 15:29:14 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:14 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:29:14 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:14 [INFO] raft: pipelining replication to peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: 2018/08/23 15:29:15 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: fatal error: concurrent map read and map write Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: goroutine 199 [running]: Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc4211659c0 sp=0xc4211659a0 pc=0x42d161 Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: runtime.mapaccess2(0x1844be0, 0xc42044dcb0, 0xc421165a88, 0x412539, 0xc42000ed40) Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc421165a08 sp=0xc4211659c0 pc=0x409c75 Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc42027bb20, 0x1d6dd40, 0xc42000ed40, 0x3085d944, 0xed31089e6, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:94 +0xe2 fp=0xc421165af8 sp=0xc421165a08 pc=0xf20ca2 Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4203f80a0, 0xc42027e580, 0x0, 0x0) Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc421165b60 sp=0xc421165af8 pc=0xf5388e Aug 23 15:29:15 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4203f80a0, 0xc42027e580) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc421165b98 sp=0xc421165b60 pc=0xf534b3 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4203e8000, 0xc42027b101, 0xc42027b180) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc421165c88 sp=0xc421165b98 pc=0xf77602 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4203e8000, 0xc420d5d140, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc421165d38 sp=0xc421165c88 pc=0xf76fff Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4203e8000, 0xc420d5d140) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc421165fa0 sp=0xc421165d38 pc=0xf76b90 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc4205d6050, 0xc4203e8000, 0xc420d5d140) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc421165fc8 sp=0xc421165fa0 pc=0xfb8e9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: runtime.goexit() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc421165fd0 sp=0xc421165fc8 pc=0x45c711 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 1 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420219260, 0xc4203e2f00) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420219260, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4202aaf00, 0xc4202aaf00, 0xc420398220, 0x37) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204d9f68) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: main.main() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 5 [syscall]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: os/signal.signal_recv(0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: os/signal.loop() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by os/signal.init.0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 93 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420c220c0, 0xc420b2a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc42054b560) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc42054b560, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420938fc0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420c36500) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42054b5c0, 0x16e4b60, 0xc420c36560) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42054b5c0, 0x16e4b60, 0xc420c36560, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420dc7000, 0x16e4b60, 0xc420c36560, 0xc420c36560, 0xc4203368e8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420dc7000, 0xc420c36560, 0x2d, 0xc422fa47c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc4203368c0, 0x1d72340, 0xc420dc7000, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc4203368c0, 0x1d72340, 0xc420dc7000, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc4203368c0, 0x1d72340, 0xc420dc7000, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc420c220c0, 0xc4203368c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 68 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc42035c300) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 22 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4200a9810, 0x1bd9776, 0x1, 0x0, 0x6) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc420132d10) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 67 [sleep]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: time.Sleep(0x3b9aca00) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420358000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 66 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc4201ffc80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 52 [select, locked to thread]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: runtime.selectgo(0xc4204b7750, 0xc42032c4e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: runtime.ensureSigM.func1() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: runtime.goexit() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 69 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4203143c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 126 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420879590) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 71 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4203664e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc420384120, 0xc420069360) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 25 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4204262c0, 0x1d71840, 0xc4201ffd00) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 92 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202324d0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 36 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7ff00, 0x72, 0xc4205459a8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc420393c18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc420393c18, 0xc420524000, 0x1000, 0x1000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Read(0xc420393c00, 0xc420524000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).Read(0xc420393c00, 0xc420524000, 0x1000, 0x1000, 0x458600, 0xc420286780, 0x4) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*conn).Read(0xc420520000, 0xc420524000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/http.(*persistConn).Read(0xc4201be240, 0xc420524000, 0x1000, 0x1000, 0xc420545b98, 0x405675, 0xc4205540c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc420522000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).Peek(0xc420522000, 0x1, 0x0, 0x0, 0x1, 0xc420f45860, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/http.(*persistConn).readLoop(0xc4201be240) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by net/http.(*Transport).dialConn Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 205 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42116a8a0, 0xc421005700, 0x20, 0x20, 0x169d4e0, 0xc421b54260) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421b66030, 0x1d718c0, 0xc42116a8a0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4203e8000, 0xc421005950, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42116a180, 0xc420d66a80, 0xc420d17bc0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: reflect.Value.call(0xc42050c840, 0xc420205558, 0x13, 0x1bda9a0, 0x4, 0xc421190d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: reflect.Value.Call(0xc42050c840, 0xc420205558, 0x13, 0xc421190d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*service).call(0xc4210e04c0, 0xc420d320f0, 0xc4205d60a0, 0x0, 0xc4211cd100, 0xc42163a100, 0x19758e0, 0xc420d66a80, 0x16, 0x169e020, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc420d320f0, 0x1d72340, 0xc4210e0500, 0x0, 0xc4202340c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc4202340c0, 0xc420d320f0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 125 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200b42c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:29:16 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc42065dc70) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 183 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42055e600, 0xc420ee2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc420d80c00) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc420d80c00, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4210306f0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420b13000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d80c60, 0x16e4b60, 0xc420b13060) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d80c60, 0x16e4b60, 0xc420b13060, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420f64980, 0x16e4b60, 0xc420b13060, 0xc420b13060, 0xc4203368e8) Aug 23 15:29:16 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420f64980, 0xc420b13060, 0x2d, 0xc4222c9e40) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc4203368c0, 0x1d72340, 0xc420f64980, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc4203368c0, 0x1d72340, 0xc420f64980, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc4203368c0, 0x1d72340, 0xc420f64980, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc42055e600, 0xc4203368c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 37 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/http.(*persistConn).writeLoop(0xc4201be240) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by net/http.(*Transport).dialConn Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 90 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7f950, 0x72, 0xc420060ce8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc420507418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc420507418, 0xc420be8000, 0x1000, 0x1000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Read(0xc420507400, 0xc420be8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:16 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).Read(0xc420507400, 0xc420be8000, 0x1000, 0x1000, 0x8e, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*conn).Read(0xc4202041f8, 0xc420be8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).Read(0xc420720ae0, 0xc420a503c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadAtLeast(0x1d59aa0, 0xc420720ae0, 0xc420a503c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadFull(0x1d59aa0, 0xc420720ae0, 0xc420a503c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202324d0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202324d0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 89 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202324d0, 0xc421197d10, 0x30, 0xf9866a) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202324d0, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420d29a80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc4202041f8, 0xc420336820) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc4202041f8, 0xc420336820) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 112 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420133130) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 142 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4202343c0, 0xc420b14000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc42054b440) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc42054b440, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420938c90, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420c36480) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42054b4a0, 0x16e4b60, 0xc420c364e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42054b4a0, 0x16e4b60, 0xc420c364e0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420dc6e00, 0x16e4b60, 0xc420c364e0, 0xc420c364e0, 0xc4205a91a8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420dc6e00, 0xc420c364e0, 0x2d, 0xc4222c9ce0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc4205a9180, 0x1d72340, 0xc420dc6e00, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc4205a9180, 0x1d72340, 0xc420dc6e00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc4205a9180, 0x1d72340, 0xc420dc6e00, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc4202343c0, 0xc4205a9180) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 124 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200b42c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc42065dc60) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 235 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4230ad3b0, 0xc4207ea000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 127 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4207bc8f0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 91 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202324d0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 88 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4203580b0, 0xc420542d10, 0x30, 0xf9866a) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4203580b0, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420d29a80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc420204160, 0xc420336780) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc420204160, 0xc420336780) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 123 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200b42c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200b42c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200b42c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc42065dc50) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 193 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4206aa320) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 202 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7f2d0, 0x72, 0xc4213fece8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc42096b998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc42096b998, 0xc420cef000, 0x1000, 0x1000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Read(0xc42096b980, 0xc420cef000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).Read(0xc42096b980, 0xc420cef000, 0x1000, 0x1000, 0xc4204b25d8, 0x1, 0xc4204b2608) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*conn).Read(0xc4202042d8, 0xc420cef000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).Read(0xc420d3a120, 0xc4205d6070, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadAtLeast(0x1d59aa0, 0xc420d3a120, 0xc4205d6070, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadFull(0x1d59aa0, 0xc420d3a120, 0xc4205d6070, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42137e000, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42137e000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 128 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7fd60, 0x72, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc420f48018, 0x72, 0xc420dd6400, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc420f48018, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Accept(0xc420f48000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).accept(0xc420f48000, 0x1, 0xc4204b8fb8, 0x639149) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*TCPListener).accept(0xc42000e690, 0x3, 0x4, 0xc4209481c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*TCPListener).AcceptTCP(0xc42000e690, 0x4, 0xc4209481c0, 0x15) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4208a43c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 129 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7fc90, 0x72, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc420f48098, 0x72, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc420f48098, 0xc421d4a000, 0x10000, 0x10000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).ReadFrom(0xc420f48080, 0xc421d4a000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).readFrom(0xc420f48080, 0xc421d4a000, 0x10000, 0x10000, 0x26, 0x26, 0x1d62b40, 0x412539, 0xc42100be40) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*UDPConn).readFrom(0xc42000e6a8, 0xc421d4a000, 0x10000, 0x10000, 0x101, 0xc42100be18, 0xc42100be90, 0x443cf7) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*UDPConn).ReadFrom(0xc42000e6a8, 0xc421d4a000, 0x10000, 0x10000, 0x10000, 0x10000, 0x56443, 0x98d2ce73, 0x2b96de0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4208a43c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 146 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4208a43c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 147 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4208a43c0, 0x12a05f200, 0xc420c40360, 0xc420f0faa0, 0xc4206343a0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:110 +0xef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 148 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4208a43c0, 0xc420f0faa0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 149 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4208a43c0, 0x1dcd6500, 0xc420c403c0, 0xc420f0faa0, 0xc4206343b0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 150 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc421246780) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 151 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc421246780) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 152 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421246780, 0x1bdd09a, 0x6, 0xc420879d10) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 153 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421246780, 0x1bdba6c, 0x5, 0xc420879d70) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 154 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421246780, 0x1bdbc6a, 0x5, 0xc420879e00) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 215 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4230b0880, 0xc421190d28, 0x20, 0x20, 0xc42043c800, 0x1bd9a98) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422431b60, 0x1d71840, 0xc4230b0880, 0xc4230caf00, 0x1849b5) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42027b180, 0xc421309f20, 0x1849b5, 0x1d71840, 0xc4230b0880, 0x0, 0xc42086f620, 0xf2c8ca, 0x1, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc420ec2140, 0x1849b5, 0xc42086f680, 0xf2c920, 0x1, 0x0, 0x0, 0xc42037eeb8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc420ec2140, 0x1849b5, 0xc4212e4120) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 156 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420c40420, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420c40420, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420c40420) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 157 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420c40480, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420c40480, 0x1dcd6500, 0xc42050af60, 0x9ebabe, 0x1, 0x0, 0x435248) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420c40480) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 158 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420c404e0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420c404e0, 0x1dcd6500, 0x1bd9928, 0x2, 0x1bd9e3a, 0x3, 0xc420d69600) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420c404e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 159 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420c40540, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420c40540, 0x1dcd6500, 0xc421182f00, 0x0, 0xc420682140, 0x3, 0x4) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420c40540) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 160 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420722e40, 0xc4203e8000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 161 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4203e8000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 162 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4203e8000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 163 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7fe30, 0x72, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc420507318, 0x72, 0xc420082000, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc420507318, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Accept(0xc420507300, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).accept(0xc420507300, 0x1, 0x1, 0xc421196f20) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*TCPListener).accept(0xc420212180, 0x406b4a, 0xc420f3f2c0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*TCPListener).Accept(0xc420212180, 0xc420f3f2c0, 0x0, 0x3f800000, 0xc420d29a80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4203e8000, 0x1d71840, 0xc420d29a80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 164 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4203f80a0, 0x3b9aca00, 0xc42035a3c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 165 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc420200f50, 0x3b9aca00, 0xc42035a3c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 166 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420360120, 0x3b9aca00, 0xc42035a3c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 167 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4203664e0, 0x3b9aca00, 0xc42035a3c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 168 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4203e8000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 169 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7fbc0, 0x72, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc420f48c18, 0x72, 0xc420426e00, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc420f48c18, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Accept(0xc420f48c00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).accept(0xc420f48c00, 0x412d88, 0x30, 0x1a1fac0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*TCPListener).accept(0xc42000ec98, 0x5fea4a, 0x1a1fac0, 0xc42051eb10) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*TCPListener).Accept(0xc42000ec98, 0xc42003e028, 0x1830320, 0x2ab0680, 0x1b7b4c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/http.(*Server).Serve(0xc420590a90, 0x1d6fc00, 0xc42000ec98, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/http.Serve(0x1d6fc00, 0xc42000ec98, 0x1d5df00, 0xc42051e5a0, 0xc4211942a0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420d29cc0, 0x1d6fc00, 0xc42000ec98, 0xc420634af0, 0xc4207230b0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 170 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc4205fda00, 0xc420634b00, 0xc420ee40c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 111 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7f540, 0x72, 0xc421163ce8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc420a62a18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc420a62a18, 0xc4211b0000, 0x1000, 0x1000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Read(0xc420a62a00, 0xc4211b0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).Read(0xc420a62a00, 0xc4211b0000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*conn).Read(0xc4202052f8, 0xc4211b0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).Read(0xc420cb1bc0, 0xc4206906d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadAtLeast(0x1d59aa0, 0xc420cb1bc0, 0xc4206906d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadFull(0x1d59aa0, 0xc420cb1bc0, 0xc4206906d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420133130, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420133130) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 101 [runnable]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: syscall.Syscall(0x0, 0xc, 0xc42078c000, 0x1000, 0xffffffffffffffff, 0x0, 0xb) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: syscall.read(0xc, 0xc42078c000, 0x1000, 0x1000, 0xc420dd6800, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/syscall/zsyscall_linux_amd64.go:749 +0x5f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: syscall.Read(0xc, 0xc42078c000, 0x1000, 0x1000, 0xbba115, 0xc420f3fce0, 0xc4211a5d38) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/syscall/syscall_unix.go:162 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Read(0xc420507380, 0xc42078c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:153 +0x118 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).Read(0xc420507380, 0xc42078c000, 0x1000, 0x1000, 0xa3, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*conn).Read(0xc420204160, 0xc42078c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).Read(0xc42068af00, 0xc420afc070, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadAtLeast(0x1d59aa0, 0xc42068af00, 0xc420afc070, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadFull(0x1d59aa0, 0xc42068af00, 0xc420afc070, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4203580b0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4203580b0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 102 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4203580b0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 103 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4203580b0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 204 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42137e000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 230 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200be2c0, 0xc420548d10, 0x30, 0xf9866a) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200be2c0, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420d29a80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc420520028, 0xc42111ee10) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc420520028, 0xc42111ee10) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 96 [runnable]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42055e480, 0xc420aa9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc42090b920) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc42090b920, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420b117a0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420b12780) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42090b980, 0x16e4b60, 0xc420b127e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42090b980, 0x16e4b60, 0xc420b127e0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420ded040, 0x16e4b60, 0xc420b127e0, 0xc420b127e0, 0xc4205a91a8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420ded040, 0xc420b127e0, 0x2d, 0xc4222c9ec0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc4205a9180, 0x1d72340, 0xc420ded040, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc4205a9180, 0x1d72340, 0xc420ded040, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc4205a9180, 0x1d72340, 0xc420ded040, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc42055e480, 0xc4205a9180) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 97 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42055e540, 0xc420cf3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc42090bc20) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc42090bc20, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420b11ec0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420b12900) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42090bc80, 0x16e4b60, 0xc420b12960) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42090bc80, 0x16e4b60, 0xc420b12960, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420ded280, 0x16e4b60, 0xc420b12960, 0xc420b12960, 0xc4203368e8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420ded280, 0xc420b12960, 0x2d, 0xc42284a240) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc4203368c0, 0x1d72340, 0xc420ded280, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc4203368c0, 0x1d72340, 0xc420ded280, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc4203368c0, 0x1d72340, 0xc420ded280, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc42055e540, 0xc4203368c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 178 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202326e0, 0xc4213f8d10, 0x30, 0xf9866a) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202326e0, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420d29a80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc420204ec8, 0xc420336dc0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc420204ec8, 0xc420336dc0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 179 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7f880, 0x72, 0xc4211a2ce8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc42096bd18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc42096bd18, 0xc420cfb000, 0x1000, 0x1000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Read(0xc42096bd00, 0xc420cfb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).Read(0xc42096bd00, 0xc420cfb000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*conn).Read(0xc420204ec8, 0xc420cfb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).Read(0xc42090bf80, 0xc420afd020, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadAtLeast(0x1d59aa0, 0xc42090bf80, 0xc420afd020, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadFull(0x1d59aa0, 0xc42090bf80, 0xc420afd020, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202326e0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202326e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 180 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202326e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 181 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202326e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 182 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4209d91a0, 0xc421107700, 0x20, 0x20, 0x169d4e0, 0xc421b54260) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421bed4a0, 0x1d718c0, 0xc4209d91a0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4203e8000, 0xc421107950, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420d3bd40, 0xc420a62980, 0xc420ea3ad0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: reflect.Value.call(0xc42050c840, 0xc4202051d0, 0x13, 0x1bda9a0, 0x4, 0xc421107d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: reflect.Value.Call(0xc42050c840, 0xc4202051d0, 0x13, 0xc421107d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*service).call(0xc420e6ab00, 0xc420336e60, 0xc420690698, 0x0, 0xc420d66600, 0xc420b12ae0, 0x19758e0, 0xc420a62980, 0x16, 0x169e020, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc420336e60, 0x1d72340, 0xc420e6ad40, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc420234600, 0xc420336e60) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 104 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420c22240, 0xc42078d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc420825800) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc420825800, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4206b7980, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207ac480) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420825860, 0x16e4b60, 0xc4207ac4e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420825860, 0x16e4b60, 0xc4207ac4e0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420d51800, 0x16e4b60, 0xc4207ac4e0, 0xc4207ac4e0, 0xc4205a91a8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420d51800, 0xc4207ac4e0, 0x2d, 0xc420657b20) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc4205a9180, 0x1d72340, 0xc420d51800, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc4205a9180, 0x1d72340, 0xc420d51800, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc4205a9180, 0x1d72340, 0xc420d51800, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc420c22240, 0xc4205a9180) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 173 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42035c180, 0xc4211c2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc420c40c00) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc420c40c00, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420723dd0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207963a0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420c40c60, 0x16e4b60, 0xc420796400) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420c40c60, 0x16e4b60, 0xc420796400, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e1c4c0, 0x16e4b60, 0xc420796400, 0xc420796400, 0xc4203368e8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e1c4c0, 0xc420796400, 0x2d, 0xc4225e6060) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc4203368c0, 0x1d72340, 0xc420e1c4c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc4203368c0, 0x1d72340, 0xc420e1c4c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc4203368c0, 0x1d72340, 0xc420e1c4c0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc42035c180, 0xc4203368c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 105 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420c22300, 0xc420bdd000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc420825c80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc420825c80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d0c180, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207ac660) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420825ce0, 0x16e4b60, 0xc4207ac6c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420825ce0, 0x16e4b60, 0xc4207ac6c0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420d51c40, 0x16e4b60, 0xc4207ac6c0, 0xc4207ac6c0, 0xc4205a91a8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420d51c40, 0xc4207ac6c0, 0x2d, 0xc4225e6180) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc4205a9180, 0x1d72340, 0xc420d51c40, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc4205a9180, 0x1d72340, 0xc420d51c40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc4205a9180, 0x1d72340, 0xc420d51c40, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc420c22300, 0xc4205a9180) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 106 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420c223c0, 0xc4211d1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc4211d6000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc4211d6000, 0x100000000000001, 0x1000000000000, 0xd40) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d0c9c0, 0x413e07) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207ac800) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4211d6060, 0x16e4b60, 0xc4207ac860) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4211d6060, 0x16e4b60, 0xc4207ac860, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420ea6580, 0x16e4b60, 0xc4207ac860, 0xc4207ac860, 0xc4203368e8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420ea6580, 0xc4207ac860, 0x2d, 0xc42019a480) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc4203368c0, 0x1d72340, 0xc420ea6580, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc4203368c0, 0x1d72340, 0xc420ea6580, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc4203368c0, 0x1d72340, 0xc420ea6580, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc420c223c0, 0xc4203368c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 174 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420132fd0, 0xc4210efd10, 0x30, 0xf9866a) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420132fd0, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420d29a80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc42000ef98, 0xc420201e00) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc42000ef98, 0xc420201e00) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 175 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7faf0, 0x72, 0xc4210eace8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc4211cd698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc4211cd698, 0xc420801000, 0x1000, 0x1000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Read(0xc4211cd680, 0xc420801000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).Read(0xc4211cd680, 0xc420801000, 0x1000, 0x1000, 0xc4210eadd8, 0x1, 0xc4210eae08) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*conn).Read(0xc42000ef98, 0xc420801000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).Read(0xc420ed8660, 0xc420712520, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadAtLeast(0x1d59aa0, 0xc420ed8660, 0xc420712520, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadFull(0x1d59aa0, 0xc420ed8660, 0xc420712520, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420132fd0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420132fd0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 176 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420132fd0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 177 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420132fd0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 194 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42055e9c0, 0xc420dee000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc420e2c9c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc420e2c9c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420e97080, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42076cda0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420e2ca20, 0x16e4b60, 0xc42076ce00) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420e2ca20, 0x16e4b60, 0xc42076ce00, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420f30c00, 0x16e4b60, 0xc42076ce00, 0xc42076ce00, 0xc420201ec8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420f30c00, 0xc42076ce00, 0x2d, 0xc420735c40) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc420201ea0, 0x1d72340, 0xc420f30c00, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc420201ea0, 0x1d72340, 0xc420f30c00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc420201ea0, 0x1d72340, 0xc420f30c00, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc42055e9c0, 0xc420201ea0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 201 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42137e000, 0xc420547d10, 0x30, 0xf9866a) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42137e000, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420d29a80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc4202042d8, 0xc420d32050) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc4202042d8, 0xc420d32050) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 203 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42137e000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 186 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42055ec00, 0xc4205ba000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc4211d76e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc4211d76e0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420ea2510, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42076d8c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4211d7740, 0x16e4b60, 0xc42076d920) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4211d7740, 0x16e4b60, 0xc42076d920, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420ea7480, 0x16e4b60, 0xc42076d920, 0xc42076d920, 0xc4205a91a8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420ea7480, 0xc42076d920, 0x2d, 0xc420f74020) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc4205a9180, 0x1d72340, 0xc420ea7480, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc4205a9180, 0x1d72340, 0xc420ea7480, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc4205a9180, 0x1d72340, 0xc420ea7480, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc42055ec00, 0xc4205a9180) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 188 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b42c0, 0xc42055ecc0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b42c0, 0xc42055ecc0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc4206fc020) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 109 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b42c0, 0xc42055ed80, 0xc420da0d80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc4207304e0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 189 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b42c0, 0xc42055ed80, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b42c0, 0xc42055ed80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc4206fc0a0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 200 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b42c0, 0xc42055ecc0, 0xc420d5d200) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc420c37f40) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 190 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420d5a180) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 191 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b42c0, 0xc42055ed80, 0x1d6eb80, 0xc420d5a180, 0xc420d3dbc0, 0xc420d3dc80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc420d5a1c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 192 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420133130, 0xc4211a3d10, 0x30, 0xf9866a) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420133130, 0x1c61988, 0xc4203e8000, 0x1d71840, 0xc420d29a80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc4202052f8, 0xc420edfd10) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d320, 0xc4202052f8, 0xc420edfd10) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 113 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420133130) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 226 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420d742a0, 0xc421305700, 0x20, 0x20, 0x169d4e0, 0xc42027b4a0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420c93e60, 0x1d718c0, 0xc420d742a0, 0x0, 0xc420d74200) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4203e8000, 0xc421305950, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4211b3980, 0xc4211c7d80, 0xc420c93d70, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: reflect.Value.call(0xc42050c840, 0xc420212b98, 0x13, 0x1bda9a0, 0x4, 0xc421305d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: reflect.Value.Call(0xc42050c840, 0xc420212b98, 0x13, 0xc421305d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*service).call(0xc420e4f6c0, 0xc420b58960, 0xc4206722c8, 0x0, 0xc4211c7800, 0xc420730720, 0x19758e0, 0xc4211c7d80, 0x16, 0x169e020, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc420b58960, 0x1d72340, 0xc420e4f740, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc42035c600, 0xc420b58960) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 227 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420556500) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 228 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b42c0, 0xc42055ecc0, 0x1d6eb80, 0xc420556500, 0xc420554240, 0xc4205542a0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b42c0, 0xc420556540) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 231 [IO wait]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.runtime_pollWait(0x7fddfdc7f3a0, 0x72, 0xc4210ebce8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).wait(0xc42148a018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*pollDesc).waitRead(0xc42148a018, 0xc42038f000, 0x1000, 0x1000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: internal/poll.(*FD).Read(0xc42148a000, 0xc42038f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*netFD).Read(0xc42148a000, 0xc42038f000, 0x1000, 0x1000, 0xc4204b45d8, 0x1, 0xc4204b4608) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net.(*conn).Read(0xc420520028, 0xc42038f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).Read(0xc4209d8360, 0xc42076ad90, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadAtLeast(0x1d59aa0, 0xc4209d8360, 0xc42076ad90, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: io.ReadFull(0x1d59aa0, 0xc4209d8360, 0xc42076ad90, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200be2c0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200be2c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 232 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200be2c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 233 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200be2c0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 234 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42035c0c0, 0xc4211db000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).fill(0xc42149ed80) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: bufio.(*Reader).ReadByte(0xc42149ed80, 0x1c66260, 0x40, 0xc4212beab0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421497230, 0x1) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421129600) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42149ede0, 0x16e4b60, 0xc421129660) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42149ede0, 0x16e4b60, 0xc421129660, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421405dc0, 0x16e4b60, 0xc421129660, 0xc421129660, 0xc42111eed8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421405dc0, 0xc421129660, 0x2d, 0xc420db8460) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequestHeader(0xc42111eeb0, 0x1d72340, 0xc421405dc0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420432fa0, 0x8) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).readRequest(0xc42111eeb0, 0x1d72340, 0xc421405dc0, 0x3, 0x3, 0xc4212bef50, 0xc4212bee18, 0xb8, 0x7, 0xc420044a70, ...) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: net/rpc.(*Server).ServeRequest(0xc42111eeb0, 0x1d72340, 0xc421405dc0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4203e8000, 0x1d71840, 0xc420d29a80, 0x1d7d0e0, 0xc42035c0c0, 0xc42111eeb0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 242 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4206aa320) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 243 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc420200f50, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4203e8000) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 244 [semacquire]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: sync.runtime_SemacquireMutex(0xc4203f812c, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: sync.(*Mutex).Lock(0xc4203f8128) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: sync.(*RWMutex).Lock(0xc4203f8128) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4203f80a0, 0x1d71840, 0xc420d29600) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 245 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420360120) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 246 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420360120) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 247 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc420bb6db0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 248 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc420d29640, 0xc4214f1ce8, 0x20, 0x20, 0x18, 0xc4230cabe0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4230ad560, 0x1d71840, 0xc420d29640, 0xc4230cabe0, 0x184987) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42027b180, 0xc421009ed0, 0x184987, 0x1d71840, 0xc420d29640, 0xc420f48260, 0xc420bb6c00, 0xc420be66f0, 0x24, 0x2bba400) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420f48200, 0x1d71840, 0xc420d29640, 0x184987, 0xc4230b07c0, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420f48200, 0x1d71840, 0xc420d29640) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 249 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc420d296c0, 0xc42100b770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4230ad980, 0x1d71840, 0xc420d296c0, 0x0, 0x1849b5) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42027b180, 0xc42100b960, 0x1849b5, 0x1d71840, 0xc420d296c0, 0x552bde, 0xc42100b950, 0xc42100b970, 0xf33ba5, 0xc42013f948) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc42013f8f0, 0x1d71840, 0xc420d296c0, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc42013f8f0) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 250 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc420d29680, 0xc42119fa78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4230ada10, 0x1d71840, 0xc420d29680, 0xc4230adb00, 0x1833a4) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42027b180, 0xc42119fc68, 0x1833a4, 0x1d71840, 0xc420d29680, 0x552bde, 0xc420200cd0, 0x2, 0xc420bf8480, 0x40) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc420d29700, 0x1833a4, 0x3b, 0xc4231edde8, 0x1, 0x1) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc420d29700) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 251 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc420d29740) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 252 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4208a45a0, 0x1d71840, 0xc420d29680) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 236 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4230ad3b0, 0xc4207ea060) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: goroutine 237 [select]: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc420ec2140) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26028]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:29:16 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:29:16 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:29:16 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: ==> Starting Nomad agent... Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: ==> Nomad agent configuration: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: Client: false Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: Log Level: INFO Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: Region: amrv01 (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: Server: true Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: Version: 0.8.4 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.472546 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: Attempting re-join to previously known node: consulnomad-02.atf01: 172.29.0.72:4648 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.472771 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [WARN] memberlist: Refuting an alive message Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.651971 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.651995 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652007 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652020 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652030 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652055 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652387 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652419 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652459 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652473 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652484 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652495 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652506 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652517 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652528 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16 [INFO] serf: Re-joined to previously known node: consulnomad-02.atf01: 172.29.0.72:4648 Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652903 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652917 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652929 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652939 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:29:16 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:16.652949 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:18 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:18 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:19 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:19 [WARN] raft: Failed to get previous log: 1598451 log not found (last: 1598449) Aug 23 15:29:19 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:19 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:29:21 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:21.261021 [ERR] http: Request /v1/agent/health?type=server, error: {"server":{"ok":false,"message":"rpc error: stream closed"}} Aug 23 15:29:22 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:22 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:29:22 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:22 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 66 Aug 23 15:29:22 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:22 [INFO] raft: Duplicate RequestVote for same term: 66 Aug 23 15:29:22 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:22 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "172.24.0.121:4647") Aug 23 15:29:22 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:22 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:29:22 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:22.820777 [INFO] server.nomad: successfully contacted 20 Nomad Servers Aug 23 15:29:23 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:23 [WARN] raft: Heartbeat timeout from "172.24.0.121:4647" reached, starting election Aug 23 15:29:23 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:23 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 67 Aug 23 15:29:23 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:23 [ERR] raft: Failed to make RequestVote RPC to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647}: EOF Aug 23 15:29:24 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:24 [INFO] raft: Duplicate RequestVote for same term: 67 Aug 23 15:29:24 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:24 [INFO] serf: EventMemberUpdate: ip-172-24-0-121.amrv01 Aug 23 15:29:25 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:25 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:26 [WARN] raft: Heartbeat timeout from "172.24.0.121:4647" reached, starting election Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:26 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 69 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:26 [INFO] raft: Duplicate RequestVote for same term: 69 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:26 [INFO] raft: Election won. Tally: 2 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:26 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:26 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:26 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:26.761173 [INFO] nomad: cluster leadership acquired Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: 2018/08/23 15:29:26 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: fatal error: concurrent map writes Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 366 [running]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc423232d08 sp=0xc423232ce8 pc=0x42d161 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: runtime.mapdelete(0x1844be0, 0xc4204a1e60, 0xc423232e20) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc423232d68 sp=0xc423232d08 pc=0x40a761 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc420502220, 0x1d6dd40, 0xc421fbc290, 0x0, 0xed3109fa6) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc423232e50 sp=0xc423232d68 pc=0xf2164c Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc420308b40, 0x1d71840, 0xc4229cebc0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc423232fc8 sp=0xc423232e50 pc=0xf56bb7 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: runtime.goexit() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc423232fd0 sp=0xc423232fc8 pc=0x45c711 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 1 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420302150, 0xc420488780) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420302150, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc42030c780, 0xc42030c780, 0xc4200751c0, 0x37) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204dbf68) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: main.main() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 5 [syscall]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: os/signal.signal_recv(0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: os/signal.loop() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by os/signal.init.0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 30 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4201f47e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 196 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42043e840) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 29 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc4202ee300) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 65 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42043e840, 0xc4211a1d10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42043e840, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc420202238, 0xc4203d1270) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc420202238, 0xc4203d1270) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 28 [sleep]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: time.Sleep(0x3b9aca00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc42043e790) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 27 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42042f580) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 15 [select, locked to thread]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: runtime.selectgo(0xc420051750, 0xc4202e29c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: runtime.ensureSigM.func1() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: runtime.goexit() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 194 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f880, 0x72, 0xc4211aace8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420f0a118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420f0a118, 0xc420f64000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420f0a100, 0xc420f64000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420f0a100, 0xc420f64000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc420202238, 0xc420f64000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc420aa7bc0, 0xc420426720, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc420aa7bc0, 0xc420426720, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc420aa7bc0, 0xc420426720, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42043e840, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42043e840) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 32 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc420506270) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0xc4200fb860, 0xc4200fb880) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4205182d0, 0xc4203147a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 72 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4201fd540, 0x1d71840, 0xc42042f5c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 195 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42043e840) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 203 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420b84000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 240 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b75518e0, 0x72, 0xc420fb2ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc421fba018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc421fba018, 0xc421eb0000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc421fba000, 0xc421eb0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc421fba000, 0xc421eb0000, 0x1000, 0x1000, 0x92, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc421fbc000, 0xc421eb0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc420cc6000, 0xc421eb21c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc420cc6000, 0xc421eb21c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc420cc6000, 0xc421eb21c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421e9a000, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421e9a000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 229 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200be2c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 197 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421369ec0, 0xc42135d700, 0x20, 0x20, 0x169d4e0, 0xc4214bba40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421ad3680, 0x1d718c0, 0xc421369ec0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420305400, 0xc42135d950, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420ddf980, 0xc420018400, 0xc4213e5ec0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.call(0xc420545f80, 0xc4202028e0, 0x13, 0x1bda9a0, 0x4, 0xc42135dd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.Call(0xc420545f80, 0xc4202028e0, 0x13, 0xc42135dd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*service).call(0xc4212687c0, 0xc4203d1310, 0xc42103e428, 0x0, 0xc420de2a00, 0xc420728560, 0x19758e0, 0xc420018400, 0x16, 0x169e020, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc4203d1310, 0x1d72340, 0xc421268800, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d0e0, 0xc420bbc6c0, 0xc4203d1310) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 241 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421e9a000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 54 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761ff00, 0x72, 0xc4200659a8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420217b18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420217b18, 0xc420574000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420217b00, 0xc420574000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420217b00, 0xc420574000, 0x1000, 0x1000, 0x458600, 0xc420577500, 0x4) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc4202026d8, 0xc420574000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/http.(*persistConn).Read(0xc4203c06c0, 0xc420574000, 0x1000, 0x1000, 0xc420065b98, 0x405675, 0xc4202e2f00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).fill(0xc420015d40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Peek(0xc420015d40, 0x1, 0x0, 0x0, 0x1, 0xc420d4c540, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/http.(*persistConn).readLoop(0xc4203c06c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by net/http.(*Transport).dialConn Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 231 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205e4370, 0xc4211a6d10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205e4370, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42025a008, 0xc4200a9090) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42025a008, 0xc4200a9090) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 55 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/http.(*persistConn).writeLoop(0xc4203c06c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by net/http.(*Transport).dialConn Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 48 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc42028e840) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc42028e840) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc42028e840) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42028e840, 0xc420597b00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 208 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4213ff140, 0xc4211bd700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422230690, 0x1d718c0, 0xc4213ff140, 0x0, 0xc4213ff100) Aug 23 15:29:26 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420305400, 0xc4211bd950, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4201f5440, 0xc420944580, 0xc422230540, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.call(0xc420545f80, 0xc420202ee0, 0x13, 0x1bda9a0, 0x4, 0xc420558d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.Call(0xc420545f80, 0xc420202ee0, 0x13, 0xc420558d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*service).call(0xc42136e400, 0xc420bf21e0, 0xc422a4e6a0, 0x0, 0xc4212c4a80, 0xc420729120, 0x19758e0, 0xc420944580, 0x16, 0x169e020, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc420bf21e0, 0x1d72340, 0xc4221a2b80, 0x0, 0xc42022af00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d0e0, 0xc42022af00, 0xc420bf21e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 36 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4205500f0, 0x1c636a0, 0xc4205e4000, 0x1d71840, 0xc4205c2940) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4205e4000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:29:26 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 304 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421e9b1e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 264 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42043e8f0, 0xc420557d10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42043e8f0, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42000fda8, 0xc421422a50) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42000fda8, 0xc421422a50) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 49 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc42028e840) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42028e840, 0xc420597b10) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 130 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc42028e840) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42028e840, 0xc420597b20) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 131 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420a9a450) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 132 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420d3a1a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 133 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761fd60, 0x72, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc4205a2d98, 0x72, 0xc42020a000, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc4205a2d98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Accept(0xc4205a2d80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).accept(0xc4205a2d80, 0xc421fbc240, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*TCPListener).accept(0xc42025a1f8, 0x458e40, 0xc4204b6f90, 0xc4204b6f98) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*TCPListener).AcceptTCP(0xc42025a1f8, 0x1c63658, 0xc420a943c0, 0xc421fbc240) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420a943c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 134 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761fc90, 0x72, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc4205a2e18, 0x72, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc4205a2e18, 0xc423568000, 0x10000, 0x10000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).ReadFrom(0xc4205a2e00, 0xc423568000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).readFrom(0xc4205a2e00, 0xc423568000, 0x10000, 0x10000, 0xc423568000, 0x10000, 0xc421fc4e60, 0x4126b9, 0xc4205aa000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*UDPConn).readFrom(0xc42025a200, 0xc423568000, 0x10000, 0x10000, 0x101, 0xc421fc4e18, 0xc421fc4e90, 0x443cf7) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*UDPConn).ReadFrom(0xc42025a200, 0xc423568000, 0x10000, 0x10000, 0x10000, 0x10000, 0xa0d4, 0x25e9ac769, 0x2b96de0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420a943c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 135 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420a943c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 136 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420a943c0, 0x12a05f200, 0xc420cc6d20, 0xc420ee4600, 0xc42035c650) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 137 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420a943c0, 0xc420ee4600) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:143 +0x1f3 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 138 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420a943c0, 0x1dcd6500, 0xc420cc6d80, 0xc420ee4600, 0xc42035c660) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 139 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4204e1b00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 140 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4204e1b00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 141 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204e1b00, 0x1bdd09a, 0x6, 0xc420a9a5a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 142 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204e1b00, 0x1bdba6c, 0x5, 0xc420a9a5d0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 143 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204e1b00, 0x1bdbc6a, 0x5, 0xc420a9a600) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 202 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f060, 0x72, 0xc420fb3ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420444a98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420444a98, 0xc420b7c000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420444a80, 0xc420b7c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420444a80, 0xc420b7c000, 0x1000, 0x1000, 0xc4211d4dd8, 0x1, 0xc4211d4e08) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc42053a0c8, 0xc420b7c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc4208342a0, 0xc421b9f690, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc4208342a0, 0xc421b9f690, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc4208342a0, 0xc421b9f690, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420b84000, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420b84000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 145 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420cc6de0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420cc6de0, 0x1dcd6500, 0x5468747541a98606, 0x7645a9a06e656b6f, 0xda6e656b6f546c61, 0x3266396136352400, 0x2d363964392d6563) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420cc6de0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 146 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4202ee840, 0xc422cdf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).fill(0xc4219ed140) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).ReadByte(0xc4219ed140, 0xc420046f00, 0xc4213eee18, 0xc4213eee20) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42304d020, 0xc42207f410) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421c9c020) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4219ed1a0, 0x16e4ba0, 0xc42304d260) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4219ed1a0, 0x16e4ba0, 0xc42304d260, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42325b6c0, 0x16e4ba0, 0xc42304d260, 0x1, 0xc42304d260) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc42325b6c0, 0xc42304d260, 0x19746e0, 0xc420db27e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc42325b6c0, 0x1be6698, 0xc, 0x19746e0, 0xc420db27e0, 0x181f280, 0xc42322a800, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc4201f47e0, 0xc4204b3e01, 0x6, 0x1d62b00, 0xc420bb0b70, 0x1, 0x1be6698, 0xc, 0x19746e0, 0xc420db27e0, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc420305400, 0xc420ec29a0, 0x1be6698, 0xc, 0x19746e0, 0xc420db27e0, 0x181f280, 0xc42322a800, 0x0, 0x1947b20) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420305400, 0x1be6698, 0xc, 0x1d76b80, 0xc420db27e0, 0x19746e0, 0xc420db27e0, 0x181f280, 0xc42322a800, 0xc4205447e0, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc42000e770, 0xc420db27e0, 0xc42322a800, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.call(0xc4205447e0, 0xc42053a2a0, 0x13, 0x1bda9a0, 0x4, 0xc421fc6d18, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.Call(0xc4205447e0, 0xc42053a2a0, 0x13, 0xc421fc6d18, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*service).call(0xc420538080, 0xc4201ff7c0, 0xc422bd80e0, 0x0, 0xc42056a980, 0xc420728080, 0x19746e0, 0xc420db27e0, 0x16, 0x181f280, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc4201ff7c0, 0x1d71b80, 0xc42322a7c0, 0x5b7ea8a6, 0xc40fce5d8b) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420305400, 0x1be6698, 0xc, 0x19746e0, 0xc420aa6ba0, 0x181f280, 0xc4212681c0, 0x1d59ee0, 0xc422848e40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420cc6e40, 0x1dcd6500, 0x44ace6881200ce78, 0x6e656d796f6c7065, 0x637345b4a0444974, 0x706d6f4364657061, 0x73616c4364657475) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:158 +0x1bb Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420cc6e40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 147 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420cc6ea0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420cc6ea0, 0x1dcd6500, 0x353a35315430322d, 0x3035372e31343a37, 0x646f4d626f4aae5a, 0x7865646e49796669, 0x654ca909851200ce) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420cc6ea0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 148 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420bbd8c0, 0xc422c9d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).fill(0xc420db2c60) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).ReadByte(0xc420db2c60, 0xc420040000, 0xc42322a718, 0xc42322a720) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4225fb4a0, 0xc42135f410) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422d72b80) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420db2cc0, 0x16e4ba0, 0xc4225fb7d0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420db2cc0, 0x16e4ba0, 0xc4225fb7d0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42322a980, 0x16e4ba0, 0xc4225fb7d0, 0x1, 0xc4225fb7d0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc42322a980, 0xc4225fb7d0, 0x19746e0, 0xc420db2b40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc42322a980, 0x1be6698, 0xc, 0x19746e0, 0xc420db2b40, 0x181f280, 0xc42322a940, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc4201f47e0, 0xc4204b3e01, 0x6, 0x1d62b00, 0xc420bb0b70, 0x1, 0x1be6698, 0xc, 0x19746e0, 0xc420db2b40, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc420305400, 0xc420ec29a0, 0x1be6698, 0xc, 0x19746e0, 0xc420db2b40, 0x181f280, 0xc42322a940, 0x0, 0x1947b20) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420305400, 0x1be6698, 0xc, 0x1d76b80, 0xc420db2b40, 0x19746e0, 0xc420db2b40, 0x181f280, 0xc42322a940, 0xc4205447e0, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc42000e770, 0xc420db2b40, 0xc42322a940, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.call(0xc4205447e0, 0xc42053a2a0, 0x13, 0x1bda9a0, 0x4, 0xc422e56d18, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.Call(0xc4205447e0, 0xc42053a2a0, 0x13, 0xc422e56d18, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*service).call(0xc420538080, 0xc4201ff7c0, 0xc422bd83c8, 0x0, 0xc42056a980, 0xc420728200, 0x19746e0, 0xc420db2b40, 0x16, 0x181f280, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc4201ff7c0, 0x1d71b80, 0xc42322a900, 0x5b7ea8a6, 0xc4134eb34c) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420305400, 0x1be6698, 0xc, 0x19746e0, 0xc420aa6300, 0x181f280, 0xc421268000, 0x1d59ee0, 0xc42001b280) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420cc6f00, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:158 +0x1bb Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420cc6f00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 149 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420c7c600, 0xc420305400) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 150 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc420305400) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 151 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc420305400) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 152 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761fe30, 0x72, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc4205a2c98, 0x72, 0xc420bec600, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc4205a2c98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Accept(0xc4205a2c80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).accept(0xc4205a2c80, 0x1, 0x1, 0xc42055af20) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*TCPListener).accept(0xc42053abc0, 0x406b4a, 0xc420ed0060, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*TCPListener).Accept(0xc42053abc0, 0xc420ed0060, 0x0, 0x3f800000, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 153 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc420308b40, 0x3b9aca00, 0xc420090600) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 154 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4201ff770, 0x3b9aca00, 0xc420090600) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 155 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420256480, 0x3b9aca00, 0xc420090600) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 156 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc420506270, 0x3b9aca00, 0xc420090600) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 157 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc420305400) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 158 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761faf0, 0x72, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420d7e718, 0x72, 0xc4203f0100, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420d7e718, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Accept(0xc420d7e700, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).accept(0xc420d7e700, 0x402037, 0xc422882800, 0x1805ec0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*TCPListener).accept(0xc42025a458, 0x458e40, 0xc4211abe90, 0xc4211abe98) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*TCPListener).Accept(0xc42025a458, 0x1c65af8, 0xc422882780, 0x1d71900, 0xc420f240c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/http.(*Server).Serve(0xc4207aac30, 0x1d6fc00, 0xc42025a458, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/http.Serve(0x1d6fc00, 0xc42025a458, 0x1d5df00, 0xc420f24000, 0x15cedc845a91e1f0, 0x309c) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420bec2c0, 0x1d6fc00, 0xc42025a458, 0xc42035d440, 0xc420c7c6c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 159 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420d7e780, 0xc42035d450, 0xc420ee5f20) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 317 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4223f6fc0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 266 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42043e8f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 265 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761fbc0, 0x72, 0xc422e57ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420217e18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420217e18, 0xc423250000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420217e00, 0xc423250000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420217e00, 0xc423250000, 0x1000, 0x1000, 0xc422e57dd8, 0x1, 0xc422e57e08) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc42000fda8, 0xc423250000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc42269ac60, 0xc422c321d0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc42269ac60, 0xc422c321d0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc42269ac60, 0xc422c321d0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42043e8f0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42043e8f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 303 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421e9b1e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 123 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f950, 0x72, 0xc4211ac9a8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420a68998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420a68998, 0xc420e8f000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420a68980, 0xc420e8f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420a68980, 0xc420e8f000, 0x1000, 0x1000, 0x458600, 0xc4201e8f00, 0x4) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc42000f350, 0xc420e8f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/http.(*persistConn).Read(0xc420d48000, 0xc420e8f000, 0x1000, 0x1000, 0xc4211acb98, 0x405675, 0xc420eaca80) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).fill(0xc420762ae0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Peek(0xc420762ae0, 0x1, 0x0, 0x0, 0x1, 0xc420ec7440, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/http.(*persistConn).readLoop(0xc420d48000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by net/http.(*Transport).dialConn Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 124 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/http.(*persistConn).writeLoop(0xc420d48000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by net/http.(*Transport).dialConn Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 207 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42022ae40, 0xc422ee6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).fill(0xc4213fed20) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).ReadByte(0xc4213fed20, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d45c80, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420728f00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4213fef00, 0x16e4b60, 0xc420728f60) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4213fef00, 0x16e4b60, 0xc420728f60, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4221a2800, 0x16e4b60, 0xc420728f60, 0xc420728f60, 0xc4210e0c58) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4221a2800, 0xc420728f60, 0x2d, 0xc422b03f80) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).readRequestHeader(0xc4210e0c30, 0x1d72340, 0xc4221a2800, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a96d0, 0x8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).readRequest(0xc4210e0c30, 0x1d72340, 0xc4221a2800, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc4210e0c30, 0x1d72340, 0xc4221a2800, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d0e0, 0xc42022ae40, 0xc4210e0c30) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 302 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f7b0, 0x72, 0xc4211954e8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc421385298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc421385298, 0xc422cde000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc421385280, 0xc422cde000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc421385280, 0xc422cde000, 0x1000, 0x1000, 0xc4211955d8, 0x1, 0xc421195608) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc421fbc238, 0xc422cde000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc4219ecae0, 0xc422bd8200, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc4219ecae0, 0xc422bd8200, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc4219ecae0, 0xc422bd8200, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421e9b1e0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421e9b1e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 214 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205e4160) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 178 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202308f0, 0xc42121fd10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202308f0, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42025a6d8, 0xc420bf20a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42025a6d8, 0xc420bf20a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 179 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761fa20, 0x72, 0xc4211a9ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420ad4498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420ad4498, 0xc420c9e000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420ad4480, 0xc420c9e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420ad4480, 0xc420c9e000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc42025a6d8, 0xc420c9e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc420195bc0, 0xc4204260f0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc420195bc0, 0xc4204260f0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc420195bc0, 0xc4204260f0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202308f0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202308f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 180 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202308f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 181 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202308f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 182 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4213fede0, 0xc421203700, 0x20, 0x20, 0x169d4e0, 0xc42036b4e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420d02630, 0x1d718c0, 0xc4213fede0, 0x0, 0xc4213fed00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420305400, 0xc421203950, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420ce9980, 0xc420f96000, 0xc420d02540, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.call(0xc420545f80, 0xc42025a9e0, 0x13, 0x1bda9a0, 0x4, 0xc421203d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.Call(0xc420545f80, 0xc42025a9e0, 0x13, 0xc421203d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*service).call(0xc420bed9c0, 0xc420bf2140, 0xc420dbc548, 0x0, 0xc420d70d80, 0xc4206da740, 0x19758e0, 0xc420f96000, 0x16, 0x169e020, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc420bf2140, 0x1d72340, 0xc420beda40, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d0e0, 0xc420bbc3c0, 0xc420bf2140) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 267 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42043e8f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 226 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200be2c0, 0xc420061d10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200be2c0, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc420202068, 0xc4210600f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc420202068, 0xc4210600f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 213 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205e4160) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 227 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f6e0, 0x72, 0xc42121cce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420a69798, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420a69798, 0xc420aa1000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420a69780, 0xc420aa1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420a69780, 0xc420aa1000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc420202068, 0xc420aa1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc4211282a0, 0xc42103e4e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc4211282a0, 0xc42103e4e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc4211282a0, 0xc42103e4e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200be2c0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200be2c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 314 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420bbd500, 0xc421a1f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).fill(0xc420db2420) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).ReadByte(0xc420db2420, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4225fa840, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422d72680) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420db2480, 0x16e4b60, 0xc422d726e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420db2480, 0x16e4b60, 0xc422d726e0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42322a500, 0x16e4b60, 0xc422d726e0, 0xc422d726e0, 0xc421422b18) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42322a500, 0xc422d726e0, 0x2d, 0xc4226278a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).readRequestHeader(0xc421422af0, 0x1d72340, 0xc42322a500, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a96d0, 0x8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).readRequest(0xc421422af0, 0x1d72340, 0xc42322a500, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc421422af0, 0x1d72340, 0xc42322a500, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d0e0, 0xc420bbd500, 0xc421422af0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 319 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc423172500) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 211 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205e4160, 0xc4213f9d10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205e4160, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42000e8f0, 0xc4210e0b90) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42000e8f0, 0xc4210e0b90) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 187 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420231d90, 0xc4213f7d10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420231d90, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42000f568, 0xc420643090) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42000f568, 0xc420643090) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 188 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f610, 0x72, 0xc420fb9ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420d7e418, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420d7e418, 0xc4208f5000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420d7e400, 0xc4208f5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420d7e400, 0xc4208f5000, 0x1000, 0x1000, 0xc420fb9dd8, 0x0, 0xc420fb9e08) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc42000f568, 0xc4208f5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc4205a9b60, 0xc42003e910, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc4205a9b60, 0xc42003e910, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc4205a9b60, 0xc42003e910, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420231d90, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420231d90) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 189 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420231d90) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 190 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420231d90) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 228 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200be2c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 223 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205e4210, 0xc42119fd10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205e4210, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc420202030, 0xc420bf2050) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc420202030, 0xc420bf2050) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 356 [runnable]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420231e40, 0xc421fc7d10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420231e40, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc4202035d0, 0xc422b168c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc4202035d0, 0xc422b168c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 212 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f470, 0x72, 0xc4213f4ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420fbfd98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420fbfd98, 0xc420e26000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420fbfd80, 0xc420e26000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420fbfd80, 0xc420e26000, 0x1000, 0x1000, 0xc4213f4dd8, 0x1, 0xc4213f4e08) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc42000e8f0, 0xc420e26000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc4210eefc0, 0xc420ad8aa0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc4210eefc0, 0xc420ad8aa0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc4210eefc0, 0xc420ad8aa0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205e4160, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205e4160) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 224 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f3a0, 0x72, 0xc420fb5ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420f1a018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420f1a018, 0xc420b8e000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420f1a000, 0xc420b8e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420f1a000, 0xc420b8e000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc420202030, 0xc420b8e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc420014060, 0xc4208565b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc420014060, 0xc4208565b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc420014060, 0xc4208565b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205e4210, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205e4210) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 225 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205e4210) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 242 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205e4210) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 234 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205e4370) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 329 [runnable]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/structs.(*Evaluation).ShouldEnqueue(0xc422424b00, 0x1a1c020) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/structs/structs.go:6594 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc420305400, 0xc42036b301, 0xc42036b320) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:298 +0x117 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc420305400, 0xc4232cd1a0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc420305400, 0xc4232cd1a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420186300, 0xc420305400, 0xc4232cd1a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 245 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205e42c0, 0xc4213fbd10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205e42c0, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc420202f30, 0xc420bf2eb0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc420202f30, 0xc420bf2eb0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 246 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f2d0, 0x72, 0xc4213f5ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc4212c5018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc4212c5018, 0xc420dc3000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc4212c5000, 0xc420dc3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc4212c5000, 0xc420dc3000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc420202f30, 0xc420dc3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc42037b980, 0xc420c57120, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc42037b980, 0xc420c57120, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc42037b980, 0xc420c57120, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205e42c0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205e42c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 247 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205e42c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 248 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205e42c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 232 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f200, 0x72, 0xc420fb4ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc421354018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc421354018, 0xc420e27000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc421354000, 0xc420e27000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc421354000, 0xc420e27000, 0x1000, 0x1000, 0xc420fb4dd8, 0x0, 0xc420fb4e08) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc42025a008, 0xc420e27000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc420087c80, 0xc4205f8470, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc420087c80, 0xc4205f8470, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc420087c80, 0xc4205f8470, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205e4370, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205e4370) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 233 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205e4370) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 201 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420b84000, 0xc42121ed10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420b84000, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42053a0c8, 0xc421061450) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42053a0c8, 0xc421061450) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 291 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4218dd080) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 204 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420b84000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 205 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420d4e240, 0xc4210e4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).fill(0xc421128c60) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).ReadByte(0xc421128c60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42073b470, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420528260) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421128cc0, 0x16e4b60, 0xc4205282c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421128cc0, 0x16e4b60, 0xc4205282c0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42106c440, 0x16e4b60, 0xc4205282c0, 0xc4205282c0, 0xc421061518) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42106c440, 0xc4205282c0, 0x2d, 0xc421fdac80) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).readRequestHeader(0xc4210614f0, 0x1d72340, 0xc42106c440, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a96d0, 0x8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).readRequest(0xc4210614f0, 0x1d72340, 0xc42106c440, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc4210614f0, 0x1d72340, 0xc42106c440, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d0e0, 0xc420d4e240, 0xc4210614f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 253 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421e9a000, 0xc421fcad10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421e9a000, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc421fbc000, 0xc4211301e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc421fbc000, 0xc4211301e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 274 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421e9a000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 206 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420bbd740, 0xc422ac3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).fill(0xc420db2a20) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).ReadByte(0xc420db2a20, 0xc420040000, 0xc42106c698, 0xc42106c6a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4225fb200, 0xc4211bb448) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422d72a60) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420db2a80, 0x16e4ba0, 0xc4225fb410) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420db2a80, 0x16e4ba0, 0xc4225fb410, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42322a840, 0x16e4ba0, 0xc4225fb410, 0x1, 0xc4225fb410) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc42322a840, 0xc4225fb410, 0x1974aa0, 0xc420db28a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc42322a840, 0x1be6a7c, 0xc, 0x1974aa0, 0xc420db28a0, 0x1695860, 0xc422d729e0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc4201f47e0, 0xc4204b3e01, 0x6, 0x1d62b00, 0xc420bb0b70, 0x1, 0x1be6a7c, 0xc, 0x1974aa0, 0xc420db28a0, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc420305400, 0xc420ec29a0, 0x1be6a7c, 0xc, 0x1974aa0, 0xc420db28a0, 0x1695860, 0xc422d729e0, 0x1947b20, 0xc4211bb8c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc420305400, 0x1be6a7c, 0xc, 0x1d76c80, 0xc420db28a0, 0x1974aa0, 0xc420db28a0, 0x1695860, 0xc422d729e0, 0xc4211bb968, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Status).Peers(0xc42000e7a8, 0xc420db28a0, 0xc422d729e0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/status_endpoint.go:64 +0xd1 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.call(0xc420545860, 0xc42025ab08, 0x13, 0x1bda9a0, 0x4, 0xc4211bbd48, 0x3, 0x3, 0x1709fe0, 0xc422d729e0, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.Call(0xc420545860, 0xc42025ab08, 0x13, 0xc4211bbd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*service).call(0xc420f8e900, 0xc4211300f0, 0xc422d40628, 0x0, 0xc4219e6000, 0xc420528540, 0x1974aa0, 0xc420db28a0, 0x16, 0x1695860, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc4211300f0, 0x1d72340, 0xc421e8f280, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d0e0, 0xc42022a3c0, 0xc4211300f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 254 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421936b40, 0xc422083700, 0x20, 0x20, 0x169d4e0, 0xc4214bba40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421f64420, 0x1d718c0, 0xc421936b40, 0x0, 0xc421936b00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420305400, 0xc422083950, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4210ee780, 0xc420fbe980, 0xc421f64330, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.call(0xc420545f80, 0xc420202bb8, 0x13, 0x1bda9a0, 0x4, 0xc422083d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.Call(0xc420545f80, 0xc420202bb8, 0x13, 0xc422083d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*service).call(0xc4213b4580, 0xc4210601e0, 0xc422dba918, 0x0, 0xc420de3000, 0xc4216e20c0, 0x19758e0, 0xc420fbe980, 0x16, 0x169e020, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc4210601e0, 0x1d72340, 0xc4219d6040, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d0e0, 0xc420d4e180, 0xc4210601e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 290 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f130, 0x72, 0xc420fb6ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420944798, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420944798, 0xc422cec000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420944780, 0xc422cec000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420944780, 0xc422cec000, 0x1000, 0x1000, 0xc420fb6dd8, 0x0, 0xc420fb6e08) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc42025af10, 0xc422cec000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc420fdf0e0, 0xc422a754c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc420fdf0e0, 0xc422a754c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc420fdf0e0, 0xc422a754c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4218dd080, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4218dd080) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 354 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b75515a0, 0x72, 0xc420fb8858) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc421385618, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc421385618, 0xc422f91000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc421385600, 0xc422f91000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc421385600, 0xc422f91000, 0x1000, 0x1000, 0xc420044a00, 0xc42322b058, 0xc42322b060) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc421fbc260, 0xc422f91000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).fill(0xc420db3320) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).ReadByte(0xc420db3320, 0x72, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4225e8060, 0xc421385618) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422d730a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420db3380, 0x16e76e0, 0xc420bea0b0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420db3380, 0x16e76e0, 0xc420bea0b0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.decodeResponse(0xc42322acc0, 0x182ba60, 0xc422574640, 0xc420db3740, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:608 +0x5e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).genericRPC(0xc4205e4000, 0xc420346840, 0x24, 0xc420289720, 0x11, 0x2d60849020db3700, 0x182ba00, 0xc420db3740, 0x182ba60, 0xc422574640, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:406 +0xf9 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).AppendEntries(0xc4205e4000, 0xc420346840, 0x24, 0xc420289720, 0x11, 0xc420db3740, 0xc422574640, 0x1, 0x3) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:379 +0x97 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc42028e840, 0xc420bbd980, 0x1863f9, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:186 +0x189 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc42028e840, 0xc420bbd980) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:132 +0x3f2 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42028e840, 0xc421c9c6a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 277 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421937440, 0xc4211bf700, 0x20, 0x20, 0x169d4e0, 0xc4214bba40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421f80000, 0x1d718c0, 0xc421937440, 0x0, 0xc421937400) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420305400, 0xc4211bf950, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420ce98c0, 0xc420fbf080, 0xc421f65dd0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.call(0xc420545f80, 0xc420203240, 0x13, 0x1bda9a0, 0x4, 0xc4211bfd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.Call(0xc420545f80, 0xc420203240, 0x13, 0xc4211bfd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*service).call(0xc42136f800, 0xc420bf2f50, 0xc422dbb478, 0x0, 0xc420fd5900, 0xc421756300, 0x19758e0, 0xc420fbf080, 0x16, 0x169e020, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc420bf2f50, 0x1d72340, 0xc421bf8480, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d0e0, 0xc4202eefc0, 0xc420bf2f50) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 209 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4218dd080, 0xc421fc8d10, 0x30, 0xf9866a) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4218dd080, 0x1c61988, 0xc420305400, 0x1d71840, 0xc420bec200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42025af10, 0xc420bf3ea0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d320, 0xc42025af10, 0xc420bf3ea0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 292 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4218dd080) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 358 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420231e40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 357 [runnable]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: syscall.Syscall(0x0, 0x1d, 0xc422efa000, 0x1000, 0xffffffffffffffff, 0x0, 0xb) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: syscall.read(0x1d, 0xc422efa000, 0x1000, 0x1000, 0xc42042f700, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/syscall/zsyscall_linux_amd64.go:749 +0x5f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: syscall.Read(0x1d, 0xc422efa000, 0x1000, 0x1000, 0xbba115, 0xc4232b2300, 0xc422e58d38) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/syscall/syscall_unix.go:162 +0x49 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc421385900, 0xc422efa000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:153 +0x118 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc421385900, 0xc422efa000, 0x1000, 0x1000, 0xc422e58dd8, 0x1, 0xc422e58e08) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc4202035d0, 0xc422efa000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc421937800, 0xc421764630, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc421937800, 0xc421764630, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc421937800, 0xc421764630, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420231e40, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420231e40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 283 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b761f540, 0x72, 0xc422e55ce8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc421f2e618, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc421f2e618, 0xc422bdf000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc421f2e600, 0xc422bdf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc421f2e600, 0xc422bdf000, 0x1000, 0x1000, 0xc422e55dd8, 0x0, 0xc422e55e08) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc42000fdc0, 0xc422bdf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).Read(0xc421b3af60, 0xc422a35850, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadAtLeast(0x1d59aa0, 0xc421b3af60, 0xc422a35850, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: io.ReadFull(0x1d59aa0, 0xc421b3af60, 0xc422a35850, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42198f290, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42198f290) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 284 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42198f290) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 285 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42198f290) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 355 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc42028e840, 0xc420bbda40, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc42028e840, 0xc420bbda40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42028e840, 0xc421c9c6c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 342 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc42028e840, 0xc420bbda40, 0xc4232b2000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42028e840, 0xc42325dd60) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 341 [IO wait]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.runtime_pollWait(0x7f61b7551810, 0x72, 0xc420554ac8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).wait(0xc420fbfc98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*pollDesc).waitRead(0xc420fbfc98, 0xc421c6f000, 0x1000, 0x1000) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: internal/poll.(*FD).Read(0xc420fbfc80, 0xc421c6f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*netFD).Read(0xc420fbfc80, 0xc421c6f000, 0x1000, 0x1000, 0xc4232ab9a0, 0xc420554be0, 0x7b9b0e) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net.(*conn).Read(0xc421fbc188, 0xc421c6f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).fill(0xc421e87560) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: bufio.(*Reader).ReadByte(0xc421e87560, 0x3, 0xc420044a70, 0xc420044a00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleCommand(0xc4205e4000, 0xc421e87560, 0xc421e875c0, 0xc420474fc0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:515 +0x43 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleConn(0xc4205e4000, 0x1d71840, 0xc4205c2940, 0x1d7d320, 0xc421fbc188) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:499 +0x23d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:477 +0x1ce Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 318 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc42028e840, 0xc420bbda40, 0x1d6eb80, 0xc4223f6fc0, 0xc4232daea0, 0xc4232daf00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42028e840, 0xc4223f7040) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 339 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421936660, 0xc42129d700, 0x20, 0x20, 0x169d4e0, 0xc4214bba40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4229e6d80, 0x1d718c0, 0xc421936660, 0x0, 0xc421936600) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420305400, 0xc42129d950, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421b3ad20, 0xc420134a80, 0xc4229e6c90, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.call(0xc420545f80, 0xc42025ad40, 0x13, 0x1bda9a0, 0x4, 0xc42129dd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: reflect.Value.Call(0xc420545f80, 0xc42025ad40, 0x13, 0xc42129dd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*service).call(0xc42325a040, 0xc421422af0, 0xc422a74738, 0x0, 0xc420726c80, 0xc42325ca80, 0x19758e0, 0xc420134a80, 0x16, 0x169e020, ...) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: net/rpc.(*Server).ServeRequest(0xc421422af0, 0x1d72340, 0xc42327d480, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420305400, 0x1d71840, 0xc420bec200, 0x1d7d0e0, 0xc420bbd080, 0xc421422af0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 330 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc42028e840, 0xc420bbd980, 0xc4232cd200) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc42028e840, 0xc422d732a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 359 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420231e40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 364 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4203095e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 363 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4203095e0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 334 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4225e9350, 0xc4232cd4a0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 365 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4201ff770, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc420305400) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 367 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420256480) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 368 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420256480) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 369 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc42258f2f0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 370 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4229cec00, 0xc421fc5ce8, 0x20, 0x20, 0x18, 0xc4230752c0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422572390, 0x1d71840, 0xc4229cec00, 0xc4230752c0, 0x184987) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42036b320, 0xc421205ed0, 0x184987, 0x1d71840, 0xc4229cec00, 0xc420e58160, 0xc42258f2c0, 0xc420738ed0, 0x24, 0x2bba400) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420e58100, 0x1d71840, 0xc4229cec00, 0x184987, 0xc4223f72c0, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420e58100, 0x1d71840, 0xc4229cec00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 371 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4229cecc0, 0xc4231e5770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422572660, 0x1d71840, 0xc4229cecc0, 0x0, 0x1849b5) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42036b320, 0xc4231e5960, 0x1849b5, 0x1d71840, 0xc4229cecc0, 0x552bde, 0xc4231e5950, 0xc4231e5970, 0xf33ba5, 0xc420306a68) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc420306a10, 0x1d71840, 0xc4229cecc0, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc420306a10) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 372 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4229cec80, 0xc423215a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4225e9440, 0x1d71840, 0xc4229cec80, 0xc4225e9560, 0x1833a4) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42036b320, 0xc423215c68, 0x1833a4, 0x1d71840, 0xc4229cec80, 0x552bde, 0xc4201ff4f0, 0x2, 0xc420c9dc80, 0x40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc4229ced40, 0x1833a4, 0x3b, 0xc423296de8, 0x1, 0x1) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc4229ced40) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 373 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc4229ced80) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 374 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420a945a0, 0x1d71840, 0xc4229cec80) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 335 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4225e9350, 0xc4232cd500) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: goroutine 336 [select]: Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4223f7380, 0xc423022d28, 0x20, 0x20, 0xc420057800, 0x1bd9a98) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4225e9410, 0x1d71840, 0xc4223f7380, 0xc422d73a60, 0x1849b5) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42036b320, 0xc423219f20, 0x1849b5, 0x1d71840, 0xc4223f7380, 0x0, 0xc4232edb00, 0xf2c8ca, 0x1, 0x0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc423172500, 0x1849b5, 0xc4232edb60, 0xf2c920, 0x1, 0x0, 0x0, 0xc4211d66b8) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc423172500, 0x1849b5, 0xc4211b2ba0) Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:29:26 ip-172-24-20-236 nomad.sh[26051]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:29:26 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:29:26 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:29:26 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: ==> Starting Nomad agent... Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: ==> Nomad agent configuration: Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: Client: false Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: Log Level: INFO Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: Region: amrv01 (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: Server: true Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: Version: 0.8.4 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.217878 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: Attempting re-join to previously known node: vol-cl-control-01.global: 172.16.9.87:4648 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.218287 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.256758 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.256779 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.256791 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.256804 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.256815 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.256825 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [WARN] memberlist: Refuting an alive message Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257047 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257096 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257111 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257123 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257842 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257856 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257867 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257878 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257889 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257899 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257909 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257919 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257930 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27.257940 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:29:27 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:27 [INFO] serf: Re-joined to previously known node: vol-cl-control-01.global: 172.16.9.87:4648 Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 70 Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [INFO] raft: Election won. Tally: 2 Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29.160848 [INFO] nomad: cluster leadership acquired Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [WARN] raft: AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} rejected, sending older logs (next: 1598458) Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [INFO] raft: pipelining replication to peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:29 ip-172-24-20-236 nomad.sh[26072]: 2018/08/23 15:29:29 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: fatal error: concurrent map read and map write Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 228 [running]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc420d549c0 sp=0xc420d549a0 pc=0x42d161 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: runtime.mapaccess2(0x1844be0, 0xc4203c4ff0, 0xc420d54a88, 0x412539, 0xc42000f3b8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc420d54a08 sp=0xc420d549c0 pc=0x409c75 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc420183ae0, 0x1d6dd40, 0xc42000f3b8, 0x22d1e4e9, 0xed3108794, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:94 +0xe2 fp=0xc420d54af8 sp=0xc420d54a08 pc=0xf20ca2 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc420481cc0, 0xc420bd0160, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc420d54b60 sp=0xc420d54af8 pc=0xf5388e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc420481cc0, 0xc420bd0160) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc420d54b98 sp=0xc420d54b60 pc=0xf534b3 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4200e1b80, 0xc420374001, 0xc420374020) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc420d54c88 sp=0xc420d54b98 pc=0xf77602 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4200e1b80, 0xc4205ac000, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc420d54d38 sp=0xc420d54c88 pc=0xf76fff Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4200e1b80, 0xc4205ac000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc420d54fa0 sp=0xc420d54d38 pc=0xf76b90 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc4203f01c0, 0xc4200e1b80, 0xc4205ac000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc420d54fc8 sp=0xc420d54fa0 pc=0xfb8e9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: runtime.goexit() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420d54fd0 sp=0xc420d54fc8 pc=0x45c711 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 1 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42030de30, 0xc420495540) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42030de30, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc420314f00, 0xc420314f00, 0xc42016cfe0, 0x37) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204dff68) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: main.main() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 5 [syscall]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: os/signal.signal_recv(0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: os/signal.loop() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by os/signal.init.0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 77 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200b4000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420748750) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 69 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4201fee10, 0xc4200681b0, 0x0, 0xc4202ea120, 0xc4202ea0c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4202e4840) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 233 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc42018ec80, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4200e1b80) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 124 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202e4b00) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 78 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc421121260) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 76 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200b4000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420748730) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 211 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fd470, 0x72, 0xc420d52ce8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc420cb8518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc420cb8518, 0xc4208e2000, 0x1000, 0x1000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Read(0xc420cb8500, 0xc4208e2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).Read(0xc420cb8500, 0xc4208e2000, 0x1000, 0x1000, 0xc4204bbdd8, 0x1, 0xc4204bbe08) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*conn).Read(0xc420202d58, 0xc4208e2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).Read(0xc4213fdf20, 0xc420c35ba0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadAtLeast(0x1d59aa0, 0xc4213fdf20, 0xc420c35ba0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadFull(0x1d59aa0, 0xc4213fdf20, 0xc420c35ba0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202e5290, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202e5290) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 136 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc42001f900, 0x1d71840, 0xc420213840) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 16 [select, locked to thread]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: runtime.selectgo(0xc4204bcf50, 0xc4203304e0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: runtime.ensureSigM.func1() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: runtime.goexit() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 50 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42001f840) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 51 [sleep]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: time.Sleep(0x3b9aca00) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc42016ed10) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 52 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc4202186c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 53 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc420180fc0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 97 [runnable]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: syscall.Syscall(0x0, 0xc, 0xc420ac6000, 0x1000, 0x97, 0x1000, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: syscall.read(0xc, 0xc420ac6000, 0x1000, 0x1000, 0xc4208ba200, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/syscall/zsyscall_linux_amd64.go:749 +0x5f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: syscall.Read(0xc, 0xc420ac6000, 0x1000, 0x1000, 0x0, 0x40, 0xc42050db18) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/syscall/syscall_unix.go:162 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Read(0xc42078a200, 0xc420ac6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:153 +0x118 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).Read(0xc42078a200, 0xc420ac6000, 0x1000, 0x1000, 0x1c60d60, 0xc42050db78, 0xc42050db68) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*conn).Read(0xc42020c028, 0xc420ac6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).Read(0xc4201da900, 0xc4205fea80, 0x97, 0x338, 0x1b70ac0, 0x7fca71879020, 0xc42050dbe7) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.(*LimitedReader).Read(0xc42069cf00, 0xc4205fea80, 0x338, 0x338, 0xc4202f5b00, 0xc420040000, 0xc42050dc68) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:446 +0x63 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bytes.(*Buffer).ReadFrom(0xc422a45490, 0x1d5bf40, 0xc42069cf00, 0x7fca71879020, 0xc422a45490, 0xc42050dc01) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bytes/buffer.go:205 +0xa0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.copyBuffer(0x1d59b20, 0xc422a45490, 0x1d5bf40, 0xc42069cf00, 0x0, 0x0, 0x0, 0x1944e20, 0x1, 0xc42069cf00) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:386 +0x31a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.Copy(0x1d59b20, 0xc422a45490, 0x1d5bf40, 0xc42069cf00, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:362 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).readData(0xc420f0c000, 0xc421208070, 0xc, 0xc, 0x0, 0x1d59aa0, 0xc4201da900, 0xc42050deb0, 0x466056) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:423 +0x148 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).handleStreamMessage(0xc420238160, 0xc421208070, 0xc, 0xc, 0xc, 0xc) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:518 +0x29b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420238160, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:469 +0x176 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420238160) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 55 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc4205521a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4203c5440, 0xc420185c10) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:29:30 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 57 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fdf00, 0x72, 0xc4205799a8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc42021a818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc42021a818, 0xc420470000, 0x1000, 0x1000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Read(0xc42021a800, 0xc420470000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).Read(0xc42021a800, 0xc420470000, 0x1000, 0x1000, 0x458600, 0xc4202f5200, 0x4) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*conn).Read(0xc42000e008, 0xc420470000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/http.(*persistConn).Read(0xc4202306c0, 0xc420470000, 0x1000, 0x1000, 0xc420579b98, 0x405675, 0xc4202ea0c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc420382060) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).Peek(0xc420382060, 0x1, 0x0, 0x0, 0x1, 0xc420e721e0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/http.(*persistConn).readLoop(0xc4202306c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by net/http.(*Transport).dialConn Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 58 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/http.(*persistConn).writeLoop(0xc4202306c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by net/http.(*Transport).dialConn Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 179 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420238160) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:30 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 208 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420fa4000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 123 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202e4b00) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 79 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420b321a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 122 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fd880, 0x72, 0xc420d49ce8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc4209b5318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc4209b5318, 0xc42089c000, 0x1000, 0x1000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Read(0xc4209b5300, 0xc42089c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:30 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).Read(0xc4209b5300, 0xc42089c000, 0x1000, 0x1000, 0xc4204bb5d8, 0x1, 0xc4204bb608) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*conn).Read(0xc420202398, 0xc42089c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).Read(0xc420382780, 0xc420b5ec20, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadAtLeast(0x1d59aa0, 0xc420382780, 0xc420b5ec20, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadFull(0x1d59aa0, 0xc420382780, 0xc420b5ec20, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202e4b00, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202e4b00) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 75 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200b4000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200b4000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200b4000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420748720) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 121 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202e4b00, 0xc420edbd10, 0x30, 0xf9866a) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202e4b00, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc42111aec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc420202398, 0xc420bf65f0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc420202398, 0xc420bf65f0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 178 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420238160) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 96 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420238160, 0xc42129cd10, 0x30, 0xf9866a) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420238160, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc42111aec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc42020c028, 0xc4205f0690) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc42020c028, 0xc4205f0690) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 139 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205b0000, 0xc420d59d10, 0x30, 0xf9866a) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205b0000, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc42111aec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc42000e8f0, 0xc4205e3900) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc42000e8f0, 0xc4205e3900) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 80 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fdd60, 0x72, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc4206aeb98, 0x72, 0xc420082200, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc4206aeb98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Accept(0xc4206aeb80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).accept(0xc4206aeb80, 0xc420202210, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*TCPListener).accept(0xc420334858, 0x458e40, 0xc4204b8f90, 0xc4204b8f98) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*TCPListener).AcceptTCP(0xc420334858, 0x1c63658, 0xc4207dc3c0, 0xc420202210) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4207dc3c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 81 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fdc90, 0x72, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc4206aee98, 0x72, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc4206aee98, 0xc420fd6000, 0x10000, 0x10000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).ReadFrom(0xc4206aee80, 0xc420fd6000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).readFrom(0xc4206aee80, 0xc420fd6000, 0x10000, 0x10000, 0x345, 0x345, 0x1d62b40, 0x412539, 0xc42144ce40) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*UDPConn).readFrom(0xc420334860, 0xc420fd6000, 0x10000, 0x10000, 0x101, 0xc42144ce18, 0xc42144ce90, 0x443cf7) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*UDPConn).ReadFrom(0xc420334860, 0xc420fd6000, 0x10000, 0x10000, 0x10000, 0x10000, 0xf833, 0x86fe280e, 0x2b96de0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4207dc3c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 130 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4207dc3c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 30 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4207dc3c0, 0x12a05f200, 0xc421287380, 0xc42097c960, 0xc420748dc0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 31 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4207dc3c0, 0xc42097c960) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 32 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4207dc3c0, 0x1dcd6500, 0xc4212873e0, 0xc42097c960, 0xc420748dd0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 33 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc420582300) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 146 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc420582300) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 147 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420582300, 0x1bdd09a, 0x6, 0xc4211213b0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 148 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420582300, 0x1bdba6c, 0x5, 0xc4211213e0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 149 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc420582300, 0x1bdbc6a, 0x5, 0xc421121410) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 244 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200be2c0, 0xc420edfd10, 0x30, 0xf9866a) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200be2c0, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc42111aec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc42000e048, 0xc4213e5bd0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc42000e048, 0xc4213e5bd0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 151 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421287440, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421287440, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421287440) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 152 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4212874a0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4212874a0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4212874a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 153 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421287500, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421287500, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421287500) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 154 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421287560, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421287560, 0x1dcd6500, 0x7fca71916f18, 0xc42060c100, 0x0, 0x0, 0xc420bf5200) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421287560) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 155 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fd610, 0x77, 0xc4204ab980) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc420c26718, 0x77, 0xc420eb3000, 0xc420eb3028, 0x436544) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitWrite(0xc420c26718, 0xc420eb3000, 0x28, 0x611b26) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).WaitWrite(0xc420c26700, 0x1c65988, 0x1d718c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:440 +0x37 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).connect(0xc420c26700, 0x1d718c0, 0xc420b8d500, 0x0, 0x0, 0x1d5c800, 0xc421da89e0, 0x0, 0x0, 0x0, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:152 +0x299 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).dial(0xc420c26700, 0x1d718c0, 0xc420b8d500, 0x1d7a200, 0x0, 0x1d7a200, 0xc420eff2f0, 0xc420eb3248, 0x62f2be) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/sock_posix.go:142 +0xe9 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.socket(0x1d718c0, 0xc420b8d500, 0x1bda392, 0x3, 0x2, 0x1, 0x0, 0x0, 0x1d7a200, 0x0, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/sock_posix.go:93 +0x1a6 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.internetSocket(0x1d718c0, 0xc420b8d500, 0x1bda392, 0x3, 0x1d7a200, 0x0, 0x1d7a200, 0xc420eff2f0, 0x1, 0x0, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/ipsock_posix.go:141 +0x129 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.doDialTCP(0x1d718c0, 0xc420b8d500, 0x1bda392, 0x3, 0x0, 0xc420eff2f0, 0x2bb8aa0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/tcpsock_posix.go:62 +0xb9 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.dialTCP(0x1d718c0, 0xc420b8d500, 0x1bda392, 0x3, 0x0, 0xc420eff2f0, 0x7bf071c62, 0x2b96de0, 0x6fc2387d3) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/tcpsock_posix.go:58 +0xe4 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.dialSingle(0x1d718c0, 0xc420b8d500, 0xc420c26300, 0x1d62b00, 0xc420eff2f0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/dial.go:547 +0x375 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.dialSerial(0x1d718c0, 0xc420b8d500, 0xc420c26300, 0xc421bcc730, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/dial.go:515 +0x22d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*Dialer).DialContext(0xc420eb3848, 0x1d71880, 0xc42003e028, 0x1bda392, 0x3, 0xc420540300, 0x10, 0x0, 0x0, 0x0, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/dial.go:397 +0x678 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*Dialer).Dial(0xc420eb3848, 0x1bda392, 0x3, 0xc420540300, 0x10, 0x20, 0x20, 0x17503e0, 0x2ade3c8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/dial.go:320 +0x75 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc4207dc3c0, 0xc420540120, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:758 +0x14e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc4207dc3c0, 0xc420540120, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc4207dc3c0, 0xc420392a00, 0x14, 0x20, 0x3, 0x3, 0xc420c35760) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc420582300, 0xc420392a00, 0x14, 0x20, 0xc420c35701, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc4200e1b80, 0xc420392a00, 0x14, 0x20, 0xc420c35760, 0x10, 0xc4207c49e0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc42001c740, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc42128d410, 0xc4200e1b80) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 156 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4200e1b80) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 157 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4200e1b80) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 158 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fde30, 0x72, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc42035df98, 0x72, 0xc4203fa100, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc42035df98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Accept(0xc42035df80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).accept(0xc42035df80, 0x1, 0x1, 0xc42129bf20) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*TCPListener).accept(0xc420334838, 0x406b4a, 0xc420f04180, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*TCPListener).Accept(0xc420334838, 0xc420f04180, 0x0, 0x3f800000, 0xc42111aec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4200e1b80, 0x1d71840, 0xc42111aec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 159 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc420481cc0, 0x3b9aca00, 0xc420330960) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 160 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc42018ec80, 0x3b9aca00, 0xc420330960) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 161 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc4200da5a0, 0x3b9aca00, 0xc420330960) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 162 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc4205521a0, 0x3b9aca00, 0xc420330960) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 163 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4200e1b80) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 164 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fdbc0, 0x72, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc42021b798, 0x72, 0xc420083900, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc42021b798, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Accept(0xc42021b780, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).accept(0xc42021b780, 0x402037, 0xc42078c8a0, 0x1805ec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*TCPListener).accept(0xc420335430, 0x458e40, 0xc420509e90, 0xc420509e98) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*TCPListener).Accept(0xc420335430, 0x1c65af8, 0xc42078c820, 0x1d71900, 0xc420b6e120) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/http.(*Server).Serve(0xc420746000, 0x1d6fc00, 0xc420335430, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/http.Serve(0x1d6fc00, 0xc420335430, 0x1d5df00, 0xc420b6e060, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc42111af80, 0x1d6fc00, 0xc420335430, 0xc420749350, 0xc42128d4d0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 165 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc42021bb80, 0xc420749360, 0xc42097d5c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 166 [sleep]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: time.Sleep(0x33e9ad84d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc42030de30, 0xc42021bb80) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 173 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4202183c0, 0xc420b65000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc4210ed440) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc4210ed440, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4208f5890, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421291760) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210ed4a0, 0x16e4b60, 0xc4212917c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210ed4a0, 0x16e4b60, 0xc4212917c0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42111b840, 0x16e4b60, 0xc4212917c0, 0xc4212917c0, 0xc4210d64d8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42111b840, 0xc4212917c0, 0x2d, 0xc42200f4c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc4210d64b0, 0x1d72340, 0xc42111b840, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42018e9b0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc4210d64b0, 0x1d72340, 0xc42111b840, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc4210d64b0, 0x1d72340, 0xc42111b840, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc4202183c0, 0xc4210d64b0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 126 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420e34000, 0xc420bc9000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc42119d140) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc42119d140, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4208c8030, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420bd2b20) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42119d1a0, 0x16e4b60, 0xc420bd2b80) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42119d1a0, 0x16e4b60, 0xc420bd2b80, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4201fca00, 0x16e4b60, 0xc420bd2b80, 0xc420bd2b80, 0xc4210d64d8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4201fca00, 0xc420bd2b80, 0x2d, 0xc420660dc0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc4210d64b0, 0x1d72340, 0xc4201fca00, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42018e9b0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc4210d64b0, 0x1d72340, 0xc4201fca00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc4210d64b0, 0x1d72340, 0xc4201fca00, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc420e34000, 0xc4210d64b0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 189 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420f0d200, 0xc420bc0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc4211f9140) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc4211f9140, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4208c9230, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420ba12e0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4211f91a0, 0x16e4b60, 0xc420ba1340) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4211f91a0, 0x16e4b60, 0xc420ba1340, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420ab9480, 0x16e4b60, 0xc420ba1340, 0xc420ba1340, 0xc4210d64d8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420ab9480, 0xc420ba1340, 0x2d, 0xc422d68640) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc4210d64b0, 0x1d72340, 0xc420ab9480, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42018e9b0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc4210d64b0, 0x1d72340, 0xc420ab9480, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc4210d64b0, 0x1d72340, 0xc420ab9480, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc420f0d200, 0xc4210d64b0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 184 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fd950, 0x72, 0xc42129d9a8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc420ed0318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc420ed0318, 0xc420b39000, 0x1000, 0x1000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Read(0xc420ed0300, 0xc420b39000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).Read(0xc420ed0300, 0xc420b39000, 0x1000, 0x1000, 0x458600, 0xc4204ab980, 0x4) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*conn).Read(0xc42020c6a8, 0xc420b39000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/http.(*persistConn).Read(0xc420b92000, 0xc420b39000, 0x1000, 0x1000, 0xc42129db98, 0x405675, 0xc420f04de0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc420bf5500) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).Peek(0xc420bf5500, 0x1, 0x0, 0x0, 0x1, 0xc420d913e0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/http.(*persistConn).readLoop(0xc420b92000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by net/http.(*Transport).dialConn Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 209 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420fa4000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 231 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4206e2820) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 185 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/http.(*persistConn).writeLoop(0xc420b92000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by net/http.(*Transport).dialConn Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 125 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42090a000, 0xc42089d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc420f1f5c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc420f1f5c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420bfb7a0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420627ba0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420f1f620, 0x16e4b60, 0xc420627c00) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420f1f620, 0x16e4b60, 0xc420627c00, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42001fac0, 0x16e4b60, 0xc420627c00, 0xc420627c00, 0xc420bf66b8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42001fac0, 0xc420627c00, 0x2d, 0xc422d68560) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc420bf6690, 0x1d72340, 0xc42001fac0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42018e9b0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc420bf6690, 0x1d72340, 0xc42001fac0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc420bf6690, 0x1d72340, 0xc42001fac0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc42090a000, 0xc420bf6690) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 167 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42016e6e0, 0xc42057cd10, 0x30, 0xf9866a) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42016e6e0, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc42111aec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc420335440, 0xc4210d6410) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc420335440, 0xc4210d6410) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 168 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fd7b0, 0x72, 0xc42129ace8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc420ada098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc420ada098, 0xc420b12000, 0x1000, 0x1000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Read(0xc420ada080, 0xc420b12000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).Read(0xc420ada080, 0xc420b12000, 0x1000, 0x1000, 0x8e, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*conn).Read(0xc420335440, 0xc420b12000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).Read(0xc421287b60, 0xc420c34530, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadAtLeast(0x1d59aa0, 0xc421287b60, 0xc420c34530, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadFull(0x1d59aa0, 0xc421287b60, 0xc420c34530, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42016e6e0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42016e6e0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 169 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42016e6e0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 170 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42016e6e0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 171 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420f0ca80, 0xc420b13000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc4210ecc60) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc4210ecc60, 0x101000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420933680, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4212912c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210eccc0, 0x16e4b60, 0xc421291320) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210eccc0, 0x16e4b60, 0xc421291320, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42111b580, 0x16e4b60, 0xc421291320, 0xc421291320, 0xc4210d64d8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42111b580, 0xc421291320, 0x2d, 0xc42077c120) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc4210d64b0, 0x1d72340, 0xc42111b580, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42018e9b0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc4210d64b0, 0x1d72340, 0xc42111b580, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc4210d64b0, 0x1d72340, 0xc42111b580, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc420f0ca80, 0xc4210d64b0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 172 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420218300, 0xc420f33000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc4210ecf60) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc4210ecf60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420933dd0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421291420) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4210ecfc0, 0x16e4b60, 0xc421291480) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4210ecfc0, 0x16e4b60, 0xc421291480, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42111b680, 0x16e4b60, 0xc421291480, 0xc421291480, 0xc4210d64d8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42111b680, 0xc421291480, 0x2d, 0xc422f21b20) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc4210d64b0, 0x1d72340, 0xc42111b680, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc4210d64b0, 0x1d72340, 0xc42111b680, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc4210d64b0, 0x1d72340, 0xc42111b680, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc420218300, 0xc4210d64b0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 177 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fd6e0, 0x72, 0xc420d48ce8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc42074f318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc42074f318, 0xc420acc000, 0x1000, 0x1000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Read(0xc42074f300, 0xc420acc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).Read(0xc42074f300, 0xc420acc000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*conn).Read(0xc420335800, 0xc420acc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).Read(0xc420dc4c60, 0xc4207c7d30, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadAtLeast(0x1d59aa0, 0xc420dc4c60, 0xc4207c7d30, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadFull(0x1d59aa0, 0xc420dc4c60, 0xc4207c7d30, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205b00b0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205b00b0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 140 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fdaf0, 0x72, 0xc420d46ce8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc420e5f698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc420e5f698, 0xc420f85000, 0x1000, 0x1000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Read(0xc420e5f680, 0xc420f85000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).Read(0xc420e5f680, 0xc420f85000, 0x1000, 0x1000, 0xc42001b5d8, 0x1, 0xc42001b608) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*conn).Read(0xc42000e8f0, 0xc420f85000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).Read(0xc420d09f80, 0xc420b08680, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadAtLeast(0x1d59aa0, 0xc420d09f80, 0xc420b08680, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadFull(0x1d59aa0, 0xc420d09f80, 0xc420b08680, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205b0000, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205b0000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 141 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205b0000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 142 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205b0000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 143 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4202189c0, 0xc420ab4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc420a82420) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc420a82420, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420391110, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4210aba60) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420a82480, 0x16e4b60, 0xc4210abac0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420a82480, 0x16e4b60, 0xc4210abac0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420b22bc0, 0x16e4b60, 0xc4210abac0, 0xc4210abac0, 0xc4205e39c8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420b22bc0, 0xc4210abac0, 0x2d, 0xc422d685e0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc4205e39a0, 0x1d72340, 0xc420b22bc0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42018e9b0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc4205e39a0, 0x1d72340, 0xc420b22bc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc4205e39a0, 0x1d72340, 0xc420b22bc0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc4202189c0, 0xc4205e39a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 207 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fd060, 0x72, 0xc42144ece8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc42078b198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc42078b198, 0xc420bab000, 0x1000, 0x1000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Read(0xc42078b180, 0xc420bab000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).Read(0xc42078b180, 0xc420bab000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*conn).Read(0xc42020c288, 0xc420bab000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).Read(0xc422154ba0, 0xc421664220, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadAtLeast(0x1d59aa0, 0xc422154ba0, 0xc421664220, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadFull(0x1d59aa0, 0xc422154ba0, 0xc421664220, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420fa4000, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420fa4000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 176 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205b00b0, 0xc420d58d10, 0x30, 0xf9866a) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205b00b0, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc42111aec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc420335800, 0xc420bf9630) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc420335800, 0xc420bf9630) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 245 [IO wait]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.runtime_pollWait(0x7fca718fd540, 0x72, 0xc42050bce8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).wait(0xc422164018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*pollDesc).waitRead(0xc422164018, 0xc420b64000, 0x1000, 0x1000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: internal/poll.(*FD).Read(0xc422164000, 0xc420b64000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).Read(0xc422164000, 0xc420b64000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*conn).Read(0xc42000e048, 0xc420b64000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).Read(0xc420a49800, 0xc421e76030, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadAtLeast(0x1d59aa0, 0xc420a49800, 0xc421e76030, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: io.ReadFull(0x1d59aa0, 0xc420a49800, 0xc421e76030, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200be2c0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200be2c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 194 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205b00b0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 195 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205b00b0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 196 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4205a8360, 0xc420ec7700, 0x20, 0x20, 0x169d4e0, 0xc4203747a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4208629f0, 0x1d718c0, 0xc4205a8360, 0x0, 0xc4205a8300) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1b80, 0xc420ec7950, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42119dda0, 0xc420a7ed80, 0xc4208628a0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: reflect.Value.call(0xc4201f4420, 0xc420335b08, 0x13, 0x1bda9a0, 0x4, 0xc420ec7d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: reflect.Value.Call(0xc4201f4420, 0xc420335b08, 0x13, 0xc420ec7d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*service).call(0xc420b23640, 0xc420bf96d0, 0xc420689f58, 0x0, 0xc420e84580, 0xc4207c47a0, 0x19758e0, 0xc420a7ed80, 0x16, 0x169e020, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc420bf96d0, 0x1d72340, 0xc420b23680, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc420f0d140, 0xc420bf96d0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 218 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420f0c000, 0xc420ac7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc422d82420) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc422d82420, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422d190e0, 0x42c30b) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421ddb040) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422d82480, 0x16e4b60, 0xc421ddb0a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422d82480, 0x16e4b60, 0xc421ddb0a0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422aa1d40, 0x16e4b60, 0xc421ddb0a0, 0xc421ddb0a0, 0xc4205f07a8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422aa1d40, 0xc421ddb0a0, 0x2d, 0xc4227aa880) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc4205f0780, 0x1d72340, 0xc422aa1d40, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42018e9b0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc4205f0780, 0x1d72340, 0xc422aa1d40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc4205f0780, 0x1d72340, 0xc422aa1d40, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc420f0c000, 0xc4205f0780) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 210 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202e5290, 0xc4213b8d10, 0x30, 0xf9866a) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202e5290, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc42111aec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc420202d58, 0xc4209c6000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc420202d58, 0xc4209c6000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 215 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420a78d40) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 212 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202e5290) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 213 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202e5290) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 214 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420219200, 0xc4208e3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc420a8c240) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc420a8c240, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a15f20, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207aa240) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420a8c2a0, 0x16e4b60, 0xc4207aa2a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420a8c2a0, 0x16e4b60, 0xc4207aa2a0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4207ff980, 0x16e4b60, 0xc4207aa2a0, 0xc4207aa2a0, 0xc4209c60c8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4207ff980, 0xc4207aa2a0, 0x2d, 0xc422f21c40) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc4209c60a0, 0x1d72340, 0xc4207ff980, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42018e9b0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc4209c60a0, 0x1d72340, 0xc4207ff980, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc4209c60a0, 0x1d72340, 0xc4207ff980, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc420219200, 0xc4209c60a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 202 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b4000, 0xc42090a6c0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b4000, 0xc42090a6c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4207abce0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 232 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4206e2820) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 192 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b4000, 0xc42090a6c0, 0xc420a80180) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4207c5900) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 203 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b4000, 0xc42090a780, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b4000, 0xc42090a780) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4207abd00) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 193 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b4000, 0xc42090a780, 0xc420a80240) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4207c5a60) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 216 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b4000, 0xc42090a780, 0x1d6eb80, 0xc420a78d40, 0xc4209b2180, 0xc4209b2300) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420a79000) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 204 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420a897c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 205 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b4000, 0xc42090a6c0, 0x1d6eb80, 0xc420a897c0, 0xc420ab74a0, 0xc420ab7620) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420a89880) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 258 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421ba4240, 0xc420ec3700, 0x20, 0x20, 0x169d4e0, 0xc4203b3a40) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4210697a0, 0x1d718c0, 0xc421ba4240, 0x0, 0xc421ba4200) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e1b80, 0xc420ec3950, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4208e5020, 0xc4214de080, 0xc4210692f0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: reflect.Value.call(0xc4201f4420, 0xc42000f198, 0x13, 0x1bda9a0, 0x4, 0xc420067d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: reflect.Value.Call(0xc4201f4420, 0xc42000f198, 0x13, 0xc420067d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*service).call(0xc42068a740, 0xc420aa5090, 0xc4224424e8, 0x0, 0xc420a07100, 0xc42230b760, 0x19758e0, 0xc4214de080, 0x16, 0x169e020, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc420aa5090, 0x1d72340, 0xc42068a780, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc420e340c0, 0xc420aa5090) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 246 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200be2c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 247 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200be2c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 248 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420218240, 0xc420e55000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc4208e40c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc4208e40c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421ae99b0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42245b960) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4208e4120, 0x16e4b60, 0xc42245b9c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4208e4120, 0x16e4b60, 0xc42245b9c0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422287500, 0x16e4b60, 0xc42245b9c0, 0xc42245b9c0, 0xc4213e5ce8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422287500, 0xc42245b9c0, 0x2d, 0xc422ca9200) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc4213e5cc0, 0x1d72340, 0xc422287500, 0x60, 0x2, 0x3, 0x100000000000003, 0xc42018e9b0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc4213e5cc0, 0x1d72340, 0xc422287500, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc4213e5cc0, 0x1d72340, 0xc422287500, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc420218240, 0xc4213e5cc0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 219 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net.(*netFD).connect.func2(0x1d718c0, 0xc420b8d500, 0xc420c26700, 0xc421c47c20, 0xc421c47b00) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:129 +0xde Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by net.(*netFD).connect Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/fd_unix.go:128 +0x286 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 220 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420fa4000, 0xc4213b5d10, 0x30, 0xf9866a) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420fa4000, 0x1c61988, 0xc4200e1b80, 0x1d71840, 0xc42111aec0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc42020c288, 0xc422af8af0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d320, 0xc42020c288, 0xc422af8af0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 259 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420218540, 0xc420db8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).fill(0xc4208e58c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: bufio.(*Reader).ReadByte(0xc4208e58c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420b1acc0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42230b8e0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4208e5920, 0x16e4b60, 0xc42230b940) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4208e5920, 0x16e4b60, 0xc42230b940, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42068a880, 0x16e4b60, 0xc42230b940, 0xc42230b940, 0xc420aa50b8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42068a880, 0xc42230b940, 0x2d, 0xc42200f1a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequestHeader(0xc420aa5090, 0x1d72340, 0xc42068a880, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42018e9b0, 0x8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).readRequest(0xc420aa5090, 0x1d72340, 0xc42068a880, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: net/rpc.(*Server).ServeRequest(0xc420aa5090, 0x1d72340, 0xc42068a880, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e1b80, 0x1d71840, 0xc42111aec0, 0x1d7d0e0, 0xc420218540, 0xc420aa5090) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 234 [semacquire]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: sync.runtime_SemacquireMutex(0xc420481d4c, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: sync.(*Mutex).Lock(0xc420481d48) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: sync.(*RWMutex).Lock(0xc420481d48) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc420481cc0, 0x1d71840, 0xc42176f040) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 235 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc4200da5a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 236 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc4200da5a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 237 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc4217f46c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 238 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42176f100, 0xc42129fce8, 0x20, 0x20, 0x18, 0xc42230be00) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421752690, 0x1d71840, 0xc42176f100, 0xc42230be00, 0x184987) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420374020, 0xc420eb5ed0, 0x184987, 0x1d71840, 0xc42176f100, 0xc4206af160, 0xc4217f4690, 0xc420c62510, 0x24, 0x2bba400) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc4206af100, 0x1d71840, 0xc42176f100, 0x184987, 0xc42068b4c0, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc4206af100, 0x1d71840, 0xc42176f100) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 239 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42176f4c0, 0xc420ffd770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4217529f0, 0x1d71840, 0xc42176f4c0, 0x0, 0x1849b5) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420374020, 0xc420ffd960, 0x1849b5, 0x1d71840, 0xc42176f4c0, 0x552bde, 0xc420ffd950, 0xc420ffd970, 0xf33ba5, 0xc42111e3d8) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc42111e380, 0x1d71840, 0xc42176f4c0, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc42111e380) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 240 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42176f480, 0xc420573a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421069e90, 0x1d71840, 0xc42176f480, 0xc421a7a0c0, 0x1833a4) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420374020, 0xc420573c68, 0x1833a4, 0x1d71840, 0xc42176f480, 0x552bde, 0xc42018ea50, 0x2, 0xc42112eac0, 0x40) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc42176f580, 0x1833a4, 0x3b, 0xc421449de8, 0x1, 0x1) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc42176f580) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 260 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4217522d0, 0xc4203ce2a0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 261 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4217522d0, 0xc4203ce300) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 241 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc42176f6c0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 274 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4207dc5a0, 0x1d71840, 0xc42176f480) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 262 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc422afe1e0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: goroutine 221 [select]: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42068b740, 0xc42144bd28, 0x20, 0x20, 0xc4204a8000, 0x1bd9a98) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420f2fb00, 0x1d71840, 0xc42068b740, 0xc42230bfc0, 0x1849b5) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420374020, 0xc42132bf20, 0x1849b5, 0x1d71840, 0xc42068b740, 0x0, 0xc4203028c0, 0xc420d495c0, 0x1d830c0, 0xc420799de0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc422afe1e0, 0x1849b5, 0x0, 0x0, 0xc42157fbc0, 0xf2b527, 0x1, 0x0) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc422afe1e0, 0x1849b5, 0xc42157fe60) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26072]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:29:30 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:29:30 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:29:30 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: ==> Starting Nomad agent... Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: ==> Nomad agent configuration: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: Client: false Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: Log Level: INFO Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: Region: amrv01 (DC: test) Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: Server: true Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: Version: 0.8.4 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:30 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:30 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:30 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:30 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:30.978123 [INFO] nomad: starting 4 scheduling worker(s) for [batch system service _core] Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:30 [INFO] serf: Attempting re-join to previously known node: consulnomad-02.aaf01: 172.26.1.191:4648 Aug 23 15:29:30 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:30.978364 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [WARN] memberlist: Refuting an alive message Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.158553 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.158575 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.158587 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.158598 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.158609 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.158908 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.158929 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.158941 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159037 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159055 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159067 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159078 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159090 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159101 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31 [INFO] serf: Re-joined to previously known node: consulnomad-02.aaf01: 172.26.1.191:4648 Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159358 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159373 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159384 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159395 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159405 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:29:31 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:31.159416 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:32 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:32 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:33 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:33 [WARN] raft: Rejecting vote request from 172.24.0.121:4647 since we have a leader: 172.24.10.89:4647 Aug 23 15:29:33 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:33 [WARN] raft: Failed to get previous log: 1598463 log not found (last: 1598461) Aug 23 15:29:34 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:34 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:35 [WARN] raft: Heartbeat timeout from "172.24.0.121:4647" reached, starting election Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:35 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 73 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:35 [INFO] raft: Election won. Tally: 2 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:35 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:35 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:35 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:35.345360 [INFO] nomad: cluster leadership acquired Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:35 [WARN] raft: AppendEntries to {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} rejected, sending older logs (next: 1598464) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: 2018/08/23 15:29:35 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: fatal error: concurrent map writes Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 312 [running]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc4211ddd08 sp=0xc4211ddce8 pc=0x42d161 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: runtime.mapdelete(0x1844be0, 0xc4201bf290, 0xc4211dde20) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc4211ddd68 sp=0xc4211ddd08 pc=0x40a761 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc4203d24a0, 0x1d6dd40, 0xc420203950, 0x0, 0xed3109faf) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc4211dde50 sp=0xc4211ddd68 pc=0xf2164c Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4203f1720, 0x1d71840, 0xc42053e700) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc4211ddfc8 sp=0xc4211dde50 pc=0xf56bb7 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: runtime.goexit() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc4211ddfd0 sp=0xc4211ddfc8 pc=0x45c711 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 1 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420314540, 0xc4204f2dc0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420314540, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc42030e640, 0xc42030e640, 0xc4202c19a0, 0x37) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204dff68) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: main.main() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 5 [syscall]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: os/signal.signal_recv(0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: os/signal.loop() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by os/signal.init.0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 278 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4218ffd90) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 223 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef2d0, 0x72, 0xc4221face8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc4223bed98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc4223bed98, 0xc420c8a000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc4223bed80, 0xc420c8a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc4223bed80, 0xc420c8a000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc420202fa8, 0xc420c8a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc422a19bc0, 0xc4225b4ca0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc422a19bc0, 0xc4225b4ca0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc422a19bc0, 0xc4225b4ca0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4206c20b0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4206c20b0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 213 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beefaf0, 0x72, 0xc420fe7ce8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc420bac598, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc420bac598, 0xc421002000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc420bac580, 0xc421002000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc420bac580, 0xc421002000, 0x1000, 0x1000, 0xc420fe7dd8, 0x0, 0xc420fe7e08) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc420202190, 0xc421002000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc420a93a40, 0xc4205b3dc0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc420a93a40, 0xc4205b3dc0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc420a93a40, 0xc4205b3dc0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421008000, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421008000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 73 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc42020b740, 0x1d71840, 0xc4203f5900) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 15 [select, locked to thread]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: runtime.selectgo(0xc420330f50, 0xc4204bc480) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: runtime.ensureSigM.func1() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: runtime.goexit() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 16 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42020b680) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 66 [sleep]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: time.Sleep(0x3b9aca00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420166c60) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 67 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc4201da600) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 68 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc420086cc0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 287 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4211b7550) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 70 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc420437a00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc4201bf6e0, 0xc42017ef60) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 288 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420440fc0, 0xc42189e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).fill(0xc421f59f80) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).ReadByte(0xc421f59f80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421799cb0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420759040) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4224a2000, 0x16e4b60, 0xc4207590a0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4224a2000, 0x16e4b60, 0xc4207590a0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420bf8e80, 0x16e4b60, 0xc4207590a0, 0xc4207590a0, 0xc42039d888) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420bf8e80, 0xc4207590a0, 0x2d, 0xc42190bd80) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).readRequestHeader(0xc42039d860, 0x1d72340, 0xc420bf8e80, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4202106e0, 0x8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).readRequest(0xc42039d860, 0x1d72340, 0xc420bf8e80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).ServeRequest(0xc42039d860, 0x1d72340, 0xc420bf8e80, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d0e0, 0xc420440fc0, 0xc42039d860) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 190 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef200, 0x72, 0xc4211dbce8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc42058be98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc42058be98, 0xc422036000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc42058be80, 0xc422036000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc42058be80, 0xc422036000, 0x1000, 0x1000, 0xc42055fdd8, 0x1, 0xc42055fe08) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc42151e038, 0xc422036000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc420d96cc0, 0xc422554340, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc420d96cc0, 0xc422554340, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc420d96cc0, 0xc422554340, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4218fee70, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4218fee70) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 193 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4201da6c0, 0xc422037000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).fill(0xc4216398c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).ReadByte(0xc4216398c0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421069500, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420722da0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421639920, 0x16e4b60, 0xc420722e00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421639920, 0x16e4b60, 0xc420722e00, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420ae6840, 0x16e4b60, 0xc420722e00, 0xc420722e00, 0xc421088988) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420ae6840, 0xc420722e00, 0x2d, 0xc4212f2640) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:35 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).readRequestHeader(0xc421088960, 0x1d72340, 0xc420ae6840, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4202106e0, 0x8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).readRequest(0xc421088960, 0x1d72340, 0xc420ae6840, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).ServeRequest(0xc421088960, 0x1d72340, 0xc420ae6840, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d0e0, 0xc4201da6c0, 0xc421088960) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 212 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef540, 0x72, 0xc420332e58) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc420bac198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc420bac198, 0xc4210dba00, 0x1, 0x1) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc420bac180, 0xc4210dbab1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc420bac180, 0xc4210dbab1, 0x1, 0x1, 0xc420e1e360, 0x0, 0xc4207d8018) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc4204e4010, 0xc4210dbab1, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*connReader).backgroundRead(0xc4210dbaa0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 192 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4218fee70) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 29 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beeff00, 0x72, 0xc4200619a8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc4204fe998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc4204fe998, 0xc4205be000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc4204fe980, 0xc4205be000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc4204fe980, 0xc4205be000, 0x1000, 0x1000, 0x458600, 0xc4204ab080, 0x4) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc42000e638, 0xc4205be000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*persistConn).Read(0xc4200cd9e0, 0xc4205be000, 0x1000, 0x1000, 0xc420061b98, 0x405675, 0xc4200906c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).fill(0xc42037d320) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Peek(0xc42037d320, 0x1, 0x0, 0x0, 0x1, 0xc420e52480, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*persistConn).readLoop(0xc4200cd9e0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by net/http.(*Transport).dialConn Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 285 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7be34470, 0x72, 0xc422f06ce8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc422c43118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc422c43118, 0xc421e25000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc422c43100, 0xc421e25000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc422c43100, 0xc421e25000, 0x1000, 0x1000, 0xc422f06dd8, 0x0, 0xc422f06e08) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc42151e9b8, 0xc421e25000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc421151920, 0xc4228f5990, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc421151920, 0xc4228f5990, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc421151920, 0xc4228f5990, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4211b7550, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4211b7550) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 30 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*persistConn).writeLoop(0xc4200cd9e0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by net/http.(*Transport).dialConn Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 276 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef6e0, 0x72, 0xc4221fcce8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc420eb3f18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc420eb3f18, 0xc4219c3000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc420eb3f00, 0xc4219c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc420eb3f00, 0xc4219c3000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc42151e3a0, 0xc4219c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc421639e00, 0xc4225543b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc421639e00, 0xc4225543b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc421639e00, 0xc4225543b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4218ffd90, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4218ffd90) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 71 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420210be0, 0x1c636a0, 0xc420166dc0, 0x1d71840, 0xc4205de200) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc420166dc0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 215 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421008000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 214 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421008000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 122 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4205ac2c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205ac2c0, 0xc4205d91b0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 262 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef950, 0x72, 0xc420577ce8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc420eb3d18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc420eb3d18, 0xc4213b7000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc420eb3d00, 0xc4213b7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc420eb3d00, 0xc4213b7000, 0x1000, 0x1000, 0x6c, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc420203450, 0xc4213b7000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc420569020, 0xc4224c86b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc420569020, 0xc4224c86b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc420569020, 0xc4224c86b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4211b6fd0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4211b6fd0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 121 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4205ac2c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4205ac2c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4205ac2c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205ac2c0, 0xc4205d91a0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 123 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4205ac2c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205ac2c0, 0xc4205d91d0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 124 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420a26270) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 125 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420a04340) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 126 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beefd60, 0x72, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc420e34018, 0x72, 0xc420082200, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc420e34018, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Accept(0xc420e34000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).accept(0xc420e34000, 0xc420203008, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*TCPListener).accept(0xc42000e5e0, 0x458e40, 0xc420050f90, 0xc420050f98) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*TCPListener).AcceptTCP(0xc42000e5e0, 0x1c63658, 0xc4208ae3c0, 0xc420203008) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4208ae3c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 127 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beefc90, 0x72, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc420e34098, 0x72, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc420e34098, 0xc423508000, 0x10000, 0x10000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).ReadFrom(0xc420e34080, 0xc423508000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).readFrom(0xc420e34080, 0xc423508000, 0x10000, 0x10000, 0xe9, 0xe9, 0x1d62b40, 0x412539, 0xc4211e1e40) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*UDPConn).readFrom(0xc42000e5e8, 0xc423508000, 0x10000, 0x10000, 0x101, 0xc4211e1e18, 0xc4211e1e90, 0x443cf7) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*UDPConn).ReadFrom(0xc42000e5e8, 0xc423508000, 0x10000, 0x10000, 0x10000, 0x10000, 0xb2ce, 0x109057757, 0x2b96de0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4208ae3c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 128 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4208ae3c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 129 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4208ae3c0, 0x12a05f200, 0xc4209f54a0, 0xc420eb84e0, 0xc4205d9990) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 130 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4208ae3c0, 0xc420eb84e0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 131 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4208ae3c0, 0x1dcd6500, 0xc4209f5500, 0xc420eb84e0, 0xc4205d99a0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 132 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc421268480) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 133 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc421268480) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 134 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421268480, 0x1bdd09a, 0x6, 0xc420a263c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 135 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421268480, 0x1bdba6c, 0x5, 0xc420a263f0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 136 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc421268480, 0x1bdbc6a, 0x5, 0xc420a26420) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 210 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4209f4d80, 0xc420c1f208, 0x20, 0x20, 0xc422327d80, 0xc4209baed0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422ac70b0, 0x1d718c0, 0xc4209f4d80, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420493400, 0xc420c1f440, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Job).List(0xc420202650, 0xc4209f4120, 0xc4209baed0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/job_endpoint.go:925 +0x2ea Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: reflect.Value.call(0xc420087e00, 0xc420202b38, 0x13, 0x1bda9a0, 0x4, 0xc4211e7898, 0x3, 0x3, 0x192b7e0, 0x19, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: reflect.Value.Call(0xc420087e00, 0xc420202b38, 0x13, 0xc420fe7898, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*service).call(0xc42020b8c0, 0xc420210a00, 0xc420ad8088, 0x0, 0xc420134c80, 0xc420746140, 0x1974e60, 0xc4209f4120, 0x16, 0x169dd20, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).ServeRequest(0xc420210a00, 0x1d71b80, 0xc42063e200, 0xc4204a8800, 0xa) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420493400, 0x1be08be, 0x8, 0x1974e60, 0xc4209f40c0, 0x169dd20, 0xc4209bae70, 0xc4209f4001, 0xc4209bae70) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc42034bdc0, 0x1be08be, 0x8, 0x1974e60, 0xc4209f40c0, 0x169dd20, 0xc4209bae70, 0x0, 0xc4204a8800) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).jobListRequest(0xc420ebdac0, 0x1d6df80, 0xc4209f4060, 0xc420953c00, 0x44774f, 0xc421269980, 0xc42046c070, 0x1d) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:33 +0xe8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest(0xc420ebdac0, 0x1d6df80, 0xc4209f4060, 0xc420953c00, 0xc42046c0a0, 0xc420fe7b58, 0x38, 0xc420fe7b50) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/job_endpoint.go:18 +0x1a6 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).JobsRequest-fm(0x1d6df80, 0xc4209f4060, 0xc420953c00, 0xc420953c00, 0xc420b60080, 0x1d, 0xbed7c80b1b127764) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:141 +0x48 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc4209f4060, 0xc420953c00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.HandlerFunc.ServeHTTP(0xc42074bfa0, 0x1d6df80, 0xc4209f4060, 0xc420953c00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*ServeMux).ServeHTTP(0xc4210da4e0, 0x1d6df80, 0xc4209f4060, 0xc420953c00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc420686000, 0xc420953c00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.HandlerFunc.ServeHTTP(0xc42091a060, 0x1d6fec0, 0xc420686000, 0xc420953c00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.serverHandler.ServeHTTP(0xc4204d6a90, 0x1d6fec0, 0xc420686000, 0xc420953c00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*conn).serve(0xc4204a63c0, 0x1d71840, 0xc42063e0c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by net/http.(*Server).Serve Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 138 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4209f5560, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4209f5560, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4209f5560) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 139 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4209f55c0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4209f55c0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4209f55c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 140 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4209f5620, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4209f5620, 0x1dcd6500, 0xc4201846e0, 0x3, 0x4, 0xc4209aa780, 0x15) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4209f5620) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 141 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4209f5680, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4209f5680, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4209f5680) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 142 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef7b0, 0x77, 0xc4202f0780) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc4224d8218, 0x77, 0xc4211e3000, 0xc4211e3028, 0x436544) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitWrite(0xc4224d8218, 0xc4211e3000, 0x28, 0x611b26) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).WaitWrite(0xc4224d8200, 0x1c65988, 0x1d718c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:440 +0x37 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).connect(0xc4224d8200, 0x1d718c0, 0xc4213e5980, 0x0, 0x0, 0x1d5c800, 0xc4206e4900, 0x0, 0x0, 0x0, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:152 +0x299 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).dial(0xc4224d8200, 0x1d718c0, 0xc4213e5980, 0x1d7a200, 0x0, 0x1d7a200, 0xc42175ad50, 0xc4211e3248, 0x62f2be) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/sock_posix.go:142 +0xe9 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.socket(0x1d718c0, 0xc4213e5980, 0x1bda392, 0x3, 0x2, 0x1, 0x0, 0x0, 0x1d7a200, 0x0, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/sock_posix.go:93 +0x1a6 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.internetSocket(0x1d718c0, 0xc4213e5980, 0x1bda392, 0x3, 0x1d7a200, 0x0, 0x1d7a200, 0xc42175ad50, 0x1, 0x0, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/ipsock_posix.go:141 +0x129 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.doDialTCP(0x1d718c0, 0xc4213e5980, 0x1bda392, 0x3, 0x0, 0xc42175ad50, 0x2bb8aa0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/tcpsock_posix.go:62 +0xb9 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.dialTCP(0x1d718c0, 0xc4213e5980, 0x1bda392, 0x3, 0x0, 0xc42175ad50, 0x808e7e739, 0x2b96de0, 0x6fc239489) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/tcpsock_posix.go:58 +0xe4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.dialSingle(0x1d718c0, 0xc4213e5980, 0xc4224d8180, 0x1d62b00, 0xc42175ad50, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/dial.go:547 +0x375 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.dialSerial(0x1d718c0, 0xc4213e5980, 0xc4224d8180, 0xc42203bf30, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/dial.go:515 +0x22d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*Dialer).DialContext(0xc4211e3848, 0x1d71880, 0xc42003e028, 0x1bda392, 0x3, 0xc4228f5940, 0x10, 0x0, 0x0, 0x0, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/dial.go:397 +0x678 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*Dialer).Dial(0xc4211e3848, 0x1bda392, 0x3, 0xc4228f5940, 0x10, 0x20, 0x20, 0x17503e0, 0x2ade3c8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/dial.go:320 +0x75 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc4208ae3c0, 0xc4228f58b0, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:758 +0x14e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc4208ae3c0, 0xc4228f58b0, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc4208ae3c0, 0xc4205c4600, 0x14, 0x20, 0x3, 0x3, 0xc4206b49a0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc421268480, 0xc4205c4600, 0x14, 0x20, 0xc4206b4901, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc420493400, 0xc4205c4600, 0x14, 0x20, 0xc4206b49a0, 0x10, 0xc420722de0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc420560f40, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc4210da420, 0xc420493400) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 143 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc420493400) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 144 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc420493400) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 145 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beefe30, 0x72, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc4205cc718, 0x72, 0xc4203f5a00, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc4205cc718, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Accept(0xc4205cc700, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).accept(0xc4205cc700, 0x1, 0x1, 0xc4211dcf20) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*TCPListener).accept(0xc420203270, 0x406b4a, 0xc420dee120, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*TCPListener).Accept(0xc420203270, 0xc420dee120, 0x0, 0x3f800000, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc420493400, 0x1d71840, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 146 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4203f1720, 0x3b9aca00, 0xc4204bc900) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 147 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4202109b0, 0x3b9aca00, 0xc4204bc900) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 148 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420076480, 0x3b9aca00, 0xc4204bc900) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 149 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc420437a00, 0x3b9aca00, 0xc4204bc900) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 150 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc420493400) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 151 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beefbc0, 0x72, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc420e34c98, 0x72, 0xc421164000, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc420e34c98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Accept(0xc420e34c80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).accept(0xc420e34c80, 0x402037, 0xc420797c00, 0x1805ec0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*TCPListener).accept(0xc42000e848, 0x458e40, 0xc4211dfe90, 0xc4211dfe98) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*TCPListener).Accept(0xc42000e848, 0x1c65af8, 0xc420797b80, 0x1d71900, 0xc42091a120) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*Server).Serve(0xc4204d6a90, 0x1d6fc00, 0xc42000e848, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.Serve(0x1d6fc00, 0xc42000e848, 0x1d5df00, 0xc42091a060, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420ebdac0, 0x1d6fc00, 0xc42000e848, 0xc4205d9fd0, 0xc4210da4e0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 152 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420e34d80, 0xc4205d9fe0, 0xc420eb91a0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 153 [sleep]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: time.Sleep(0x616abf846) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc420314540, 0xc420e34d80) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 183 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421008000, 0xc4211d2d10, 0x30, 0xf9866a) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421008000, 0x1c61988, 0xc420493400, 0x1d71840, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc420202190, 0xc420afbbd0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc420202190, 0xc420afbbd0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 77 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beefa20, 0x72, 0xc4200659a8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc420214698, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc420214698, 0xc420cd8000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc420214680, 0xc420cd8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc420214680, 0xc420cd8000, 0x1000, 0x1000, 0x458600, 0xc4202f0780, 0x4) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc421054028, 0xc420cd8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*persistConn).Read(0xc421048000, 0xc420cd8000, 0x1000, 0x1000, 0xc420065b98, 0x405675, 0xc420e682a0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).fill(0xc420934960) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Peek(0xc420934960, 0x1, 0x0, 0x0, 0x1, 0xc420dc4f60, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*persistConn).readLoop(0xc421048000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by net/http.(*Transport).dialConn Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 78 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*persistConn).writeLoop(0xc421048000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by net/http.(*Transport).dialConn Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 238 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef130, 0x72, 0xc422f09ce8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc42268e018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc42268e018, 0xc421ee2000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc42268e000, 0xc421ee2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc42268e000, 0xc421ee2000, 0x1000, 0x1000, 0xc42032edd8, 0x1, 0xc42032ee08) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc421054668, 0xc421ee2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc4217682a0, 0xc42264c0e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc4217682a0, 0xc42264c0e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc4217682a0, 0xc42264c0e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4206c2160, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4206c2160) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 81 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420234420, 0xc4211d4d10, 0x30, 0xf9866a) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420234420, 0x1c61988, 0xc420493400, 0x1d71840, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc42000e868, 0xc4203cc640) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc42000e868, 0xc4203cc640) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 194 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef880, 0x72, 0xc4211dace8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc420e34e98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc420e34e98, 0xc420cdb000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc420e34e80, 0xc420cdb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc420e34e80, 0xc420cdb000, 0x1000, 0x1000, 0xc4211dadd8, 0x0, 0xc4211dae08) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc42000e868, 0xc420cdb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc4209f5b00, 0xc420a31af0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc4209f5b00, 0xc420a31af0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc4209f5b00, 0xc420a31af0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420234420, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420234420) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 195 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420234420) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 196 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420234420) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 182 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef610, 0x72, 0xc4211d0658) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc420bac098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc420bac098, 0xc4209ba200, 0x1, 0x1) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc420bac080, 0xc4209ba251, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc420bac080, 0xc4209ba251, 0x1, 0x1, 0xc4211d07a0, 0x0, 0xc4211a0060) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc4204e4000, 0xc4209ba251, 0x1, 0x1, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*connReader).backgroundRead(0xc4209ba240) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:668 +0x5a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by net/http.(*connReader).startBackgroundRead Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:664 +0xce Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 228 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420d08000, 0xc420067d10, 0x30, 0xf9866a) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420d08000, 0x1c61988, 0xc420493400, 0x1d71840, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc421054008, 0xc421833e50) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc421054008, 0xc421833e50) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 211 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420934420, 0xc420c232c8, 0x20, 0x20, 0x5, 0x1bd9a98) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420c8e900, 0x1d718c0, 0xc420934420, 0x0, 0xc420934400) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420493400, 0xc420c23500, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Node).List(0xc4205a2d20, 0xc420d99500, 0xc4210dbe60, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:1198 +0x27c Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: reflect.Value.call(0xc4205a3020, 0xc4202031c0, 0x13, 0x1bda9a0, 0x4, 0xc4211e58f8, 0x3, 0x3, 0x192ba60, 0x19, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: reflect.Value.Call(0xc4205a3020, 0xc4202031c0, 0x13, 0xc4211d28f8, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*service).call(0xc42020bbc0, 0xc420210a00, 0xc420b72280, 0x0, 0xc4205cc400, 0xc4207461c0, 0x1975760, 0xc420d99500, 0x16, 0x169e120, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).ServeRequest(0xc420210a00, 0x1d71b80, 0xc420b8e500, 0xc4204a8400, 0xa) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc420493400, 0x1be205d, 0x9, 0x1975760, 0xc420d994a0, 0x169e120, 0xc4210dbcb0, 0xc420d99401, 0xc4210dbcb0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*Agent).RPC(0xc42034bdc0, 0x1be205d, 0x9, 0x1975760, 0xc420d994a0, 0x169e120, 0xc4210dbcb0, 0xc4203693b0, 0x1e) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:833 +0x7a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest(0xc420ebdac0, 0x1d6df80, 0xc420d99440, 0xc420e3be00, 0xc4203693e0, 0xc4211d2b58, 0x38, 0xc4211d2b50) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/node_endpoint.go:24 +0x1b3 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).NodesRequest-fm(0x1d6df80, 0xc420d99440, 0xc420e3be00, 0xc420e3be00, 0xc420e20bc0, 0x1e, 0xbed7c80b1b11f488) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:145 +0x48 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/command/agent.(*HTTPServer).wrap.func1(0x1d6df80, 0xc420d99440, 0xc420e3be00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:283 +0x14c Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.HandlerFunc.ServeHTTP(0xc4206ee000, 0x1d6df80, 0xc420d99440, 0xc420e3be00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*ServeMux).ServeHTTP(0xc4210da4e0, 0x1d6df80, 0xc420d99440, 0xc420e3be00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:2337 +0x130 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1d6fec0, 0xc42074f260, 0xc420e3be00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/NYTimes/gziphandler/gzip.go:269 +0x179 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.HandlerFunc.ServeHTTP(0xc42091a060, 0x1d6fec0, 0xc42074f260, 0xc420e3be00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:1947 +0x44 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.serverHandler.ServeHTTP(0xc4204d6a90, 0x1d6fec0, 0xc42074f260, 0xc420e3be00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:2694 +0xbc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/http.(*conn).serve(0xc4204a6460, 0x1d71840, 0xc420b8e380) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:1830 +0x651 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by net/http.(*Server).Serve Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/http/server.go:2795 +0x27b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 286 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4211b7550) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 279 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4224a2480, 0xc4211e7700, 0x20, 0x20, 0x169d4e0, 0xc421191580) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4217aa180, 0x1d718c0, 0xc4224a2480, 0x0, 0xc4224a2400) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420493400, 0xc4211e7950, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc421984a20, 0xc422418180, 0xc420f2be60, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: reflect.Value.call(0xc4205a2f60, 0xc42151e6a8, 0x13, 0x1bda9a0, 0x4, 0xc4211e7d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: reflect.Value.Call(0xc4205a2f60, 0xc42151e6a8, 0x13, 0xc4211e7d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*service).call(0xc420abbb00, 0xc421088d70, 0xc42268d098, 0x0, 0xc422c42280, 0xc420723500, 0x19758e0, 0xc422418180, 0x16, 0x169e020, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).ServeRequest(0xc421088d70, 0x1d72340, 0xc420abbb80, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d0e0, 0xc42144ed80, 0xc421088d70) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 243 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).connect.func2(0x1d718c0, 0xc4213e5980, 0xc4224d8200, 0xc42075e360, 0xc42075e300) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:129 +0xde Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by net.(*netFD).connect Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:128 +0x286 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 205 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4206c20b0, 0xc420fe6d10, 0x30, 0xf9866a) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4206c20b0, 0x1c61988, 0xc420493400, 0x1d71840, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc420202fa8, 0xc4230c0e60) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc420202fa8, 0xc4230c0e60) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 277 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4218ffd90) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 229 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef470, 0x72, 0xc4221fbce8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc4205cc898, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc4205cc898, 0xc421003000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc4205cc880, 0xc421003000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc4205cc880, 0xc421003000, 0x1000, 0x1000, 0xc4221fbdd8, 0x1, 0xc4221fbe08) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc421054008, 0xc421003000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc421af1f20, 0xc4206dc030, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc421af1f20, 0xc4206dc030, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc421af1f20, 0xc4206dc030, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420d08000, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420d08000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 230 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420d08000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 231 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420d08000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 232 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420440000, 0xc420c16000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).fill(0xc42037cea0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).ReadByte(0xc42037cea0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420b56f60, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42164b200) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc42037d380, 0x16e4b60, 0xc42164b260) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc42037d380, 0x16e4b60, 0xc42164b260, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4207d92c0, 0x16e4b60, 0xc42164b260, 0xc42164b260, 0xc421833f18) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4207d92c0, 0xc42164b260, 0x2d, 0xc4223f5ce0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).readRequestHeader(0xc421833ef0, 0x1d72340, 0xc4207d92c0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4202106e0, 0x8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).readRequest(0xc421833ef0, 0x1d72340, 0xc4207d92c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).ServeRequest(0xc421833ef0, 0x1d72340, 0xc4207d92c0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d0e0, 0xc420440000, 0xc421833ef0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 220 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7beef3a0, 0x72, 0xc420febce8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc42077ba98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc42077ba98, 0xc4213bc000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc42077ba80, 0xc4213bc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc42077ba80, 0xc4213bc000, 0x1000, 0x1000, 0xc420febdd8, 0x0, 0xc420febe08) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc4204e5618, 0xc4213bc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc4214412c0, 0xc42077d140, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc4214412c0, 0xc42077d140, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc4214412c0, 0xc42077d140, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4221a0210, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4221a0210) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 221 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4221a0210) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 222 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4221a0210) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 239 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4206c2160) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 224 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4206c20b0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 225 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4206c20b0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 242 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420b0d920, 0xc4211e5700, 0x20, 0x20, 0x169d4e0, 0xc421191580) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4210796e0, 0x1d718c0, 0xc420b0d920, 0x0, 0xc420b0d900) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420493400, 0xc4211e5950, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420b0cf60, 0xc4204fe480, 0xc421079260, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: reflect.Value.call(0xc4205a2f60, 0xc42000ec88, 0x13, 0x1bda9a0, 0x4, 0xc4211e5d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: reflect.Value.Call(0xc4205a2f60, 0xc42000ec88, 0x13, 0xc4211e5d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*service).call(0xc4208a87c0, 0xc4224af770, 0xc42093f5d0, 0x0, 0xc42097b880, 0xc42067e320, 0x19758e0, 0xc4204fe480, 0x16, 0x169e020, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).ServeRequest(0xc4224af770, 0x1d72340, 0xc4208a8840, 0x0, 0xc4201da180) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d0e0, 0xc4201da180, 0xc4224af770) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 191 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4218fee70) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 275 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4218ffd90, 0xc420fe5d10, 0x30, 0xf9866a) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4218ffd90, 0x1c61988, 0xc420493400, 0x1d71840, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc42151e3a0, 0xc421088cd0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc42151e3a0, 0xc421088cd0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 237 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4206c2160, 0xc422f0dd10, 0x30, 0xf9866a) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4206c2160, 0x1c61988, 0xc420493400, 0x1d71840, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc421054668, 0xc4203cd860) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc421054668, 0xc4203cd860) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 189 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4218fee70, 0xc4211d9d10, 0x30, 0xf9866a) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4218fee70, 0x1c61988, 0xc420493400, 0x1d71840, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc42151e038, 0xc4210888c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc42151e038, 0xc4210888c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 263 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4211b6fd0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 264 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4211b6fd0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 240 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4206c2160) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 241 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420440f00, 0xc421ee3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).fill(0xc421dfaa80) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).ReadByte(0xc421dfaa80, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4215a88d0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42067ec00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421dfaae0, 0x16e4b60, 0xc42067ecc0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421dfaae0, 0x16e4b60, 0xc42067ecc0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420baa440, 0x16e4b60, 0xc42067ecc0, 0xc42067ecc0, 0xc4203cd928) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420baa440, 0xc42067ecc0, 0x2d, 0xc420b45fc0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).readRequestHeader(0xc4203cd900, 0x1d72340, 0xc420baa440, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4202106e0, 0x8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).readRequest(0xc4203cd900, 0x1d72340, 0xc420baa440, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).ServeRequest(0xc4203cd900, 0x1d72340, 0xc420baa440, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d0e0, 0xc420440f00, 0xc4203cd900) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 282 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7be34540, 0x72, 0xc4221f6858) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc422c42d98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc422c42d98, 0xc42249e000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc422c42d80, 0xc42249e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc422c42d80, 0xc42249e000, 0x1000, 0x1000, 0xc420044a00, 0xc420bab358, 0xc420bab360) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc42000ed20, 0xc42249e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).fill(0xc4213e4540) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).ReadByte(0xc4213e4540, 0x72, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42125e480, 0xc422c42d98) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420770160) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4213e45a0, 0x16e76e0, 0xc42194ca50) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4213e45a0, 0x16e76e0, 0xc42194ca50, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.decodeResponse(0xc420bd20c0, 0x182ba60, 0xc420b452e0, 0xc421dfb0e0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:608 +0x5e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).genericRPC(0xc420166dc0, 0xc4200a24e0, 0x24, 0xc42028be60, 0x11, 0x1496779021dfb000, 0x182ba00, 0xc421dfb0e0, 0x182ba60, 0xc420b452e0, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:406 +0xf9 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).AppendEntries(0xc420166dc0, 0xc4200a24e0, 0x24, 0xc42028be60, 0x11, 0xc421dfb0e0, 0xc420b452e0, 0x1, 0x3) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:379 +0x97 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4205ac2c0, 0xc4201dab40, 0x186401, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:186 +0x189 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4205ac2c0, 0xc4201dab40) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:132 +0x3f2 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205ac2c0, 0xc420758a20) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 283 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4205ac2c0, 0xc4201dac00, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4205ac2c0, 0xc4201dac00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205ac2c0, 0xc420758a40) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 284 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4211b7550, 0xc420fead10, 0x30, 0xf9866a) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4211b7550, 0x1c61988, 0xc420493400, 0x1d71840, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc42151e9b8, 0xc42039d770) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc42151e9b8, 0xc42039d770) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 244 [runnable]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4203f1720, 0xc42002f1e0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:210 +0x359 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4203f1720, 0xc42002f1e0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc420493400, 0xc4201bc001, 0xc4201bc000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc420493400, 0xc42075e3c0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc420493400, 0xc42075e3c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc4201a2090, 0xc420493400, 0xc42075e3c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 293 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4205ac2c0, 0xc4201dab40, 0xc420755980) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205ac2c0, 0xc42067f360) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 245 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4205ac2c0, 0xc4201dac00, 0xc42075e420) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205ac2c0, 0xc4207711c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 267 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42173e840, 0xc422c41d10, 0x30, 0xf9866a) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42173e840, 0x1c61988, 0xc420493400, 0x1d71840, 0xc420ebd9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc420203498, 0xc421c4a5f0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d320, 0xc420203498, 0xc421c4a5f0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 268 [IO wait]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.runtime_pollWait(0x7f7e7be343a0, 0x72, 0xc422f07ce8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).wait(0xc42212a398, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*pollDesc).waitRead(0xc42212a398, 0xc420c86000, 0x1000, 0x1000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: internal/poll.(*FD).Read(0xc42212a380, 0xc420c86000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*netFD).Read(0xc42212a380, 0xc420c86000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net.(*conn).Read(0xc420203498, 0xc420c86000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).Read(0xc420569ec0, 0xc4228f5a30, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadAtLeast(0x1d59aa0, 0xc420569ec0, 0xc4228f5a30, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: io.ReadFull(0x1d59aa0, 0xc420569ec0, 0xc4228f5a30, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42173e840, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42173e840) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 269 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42173e840) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 270 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42173e840) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 271 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420441080, 0xc420c87000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).fill(0xc4226304e0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: bufio.(*Reader).ReadByte(0xc4226304e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc421935bc0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42074b640) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422630540, 0x16e4b60, 0xc42074b720) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422630540, 0x16e4b60, 0xc42074b720, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4206d2c40, 0x16e4b60, 0xc42074b720, 0xc42074b720, 0xc421c4a6b8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4206d2c40, 0xc42074b720, 0x2d, 0xc42190a340) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).readRequestHeader(0xc421c4a690, 0x1d72340, 0xc4206d2c40, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4202106e0, 0x8) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).readRequest(0xc421c4a690, 0x1d72340, 0xc4206d2c40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: net/rpc.(*Server).ServeRequest(0xc421c4a690, 0x1d72340, 0xc4206d2c40, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420493400, 0x1d71840, 0xc420ebd9c0, 0x1d7d0e0, 0xc420441080, 0xc421c4a690) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 294 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4219682a0, 0xc4201bbc20) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 307 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc42053e080) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 308 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4205ac2c0, 0xc4201dac00, 0x1d6eb80, 0xc42053e080, 0xc420774240, 0xc420774480) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4205ac2c0, 0xc42053e1c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 309 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc42030df40) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 310 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc42030df40) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 311 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4202109b0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc420493400) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 313 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420076480) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 314 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420076480) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 315 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc42197a210) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 316 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42053e840, 0xc422dd3ce8, 0x20, 0x20, 0x18, 0xc42067f9c0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4219686c0, 0x1d71840, 0xc42053e840, 0xc42067f9c0, 0x184987) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4201bc000, 0xc422d59ed0, 0x184987, 0x1d71840, 0xc42053e840, 0xc420e34560, 0xc42197a1b0, 0xc4207a6930, 0x24, 0x2bba400) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420e34500, 0x1d71840, 0xc42053e840, 0x184987, 0xc420babf80, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420e34500, 0x1d71840, 0xc42053e840) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 317 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42053ebc0, 0xc4211c5770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42197b530, 0x1d71840, 0xc42053ebc0, 0x0, 0x1849b5) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4201bc000, 0xc4211c5960, 0x1849b5, 0x1d71840, 0xc42053ebc0, 0x552bde, 0xc4211c5950, 0xc4211c5970, 0xf33ba5, 0xc422e00c28) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc422e00bd0, 0x1d71840, 0xc42053ebc0, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc422e00bd0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 318 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42053e980, 0xc42111da78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4218e9a70, 0x1d71840, 0xc42053e980, 0xc4218e9ce0, 0x1833a4) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4201bc000, 0xc42111dc68, 0x1833a4, 0x1d71840, 0xc42053e980, 0x552bde, 0xc420210780, 0x2, 0xc420783f40, 0x40) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc42053ed80, 0x1833a4, 0x3b, 0xc422dd7de8, 0x1, 0x1) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc42053ed80) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 319 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc42053ee80) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 320 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4208ae5a0, 0x1d71840, 0xc42053e980) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 295 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc4219682a0, 0xc4203c2000) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 296 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc422bb6e60) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: goroutine 289 [select]: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc4203cc3c0, 0x7f7e781d02f8, 0xc42067c100, 0x1, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:248 +0x2c1 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc4203cc3c0, 0x7f7e781d02f8, 0xc42067c100, 0x7f7e781d02f8, 0xc42067c100) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsImpl(0xc422bb6e60, 0xc4219922a0, 0xc4206d0840, 0x9d78a50294606ea0, 0xc423158e28, 0x9d78a50294606ea0, 0x1afe340, 0xc4219922b0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:756 +0x89 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).(github.com/hashicorp/nomad/nomad/deploymentwatcher.getAllocsImpl)-fm(0xc4219922a0, 0xc4206d0840, 0xc4201bbda0, 0xc421cb0318, 0xc4204bc700, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x3e Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4201bc000, 0xc422dd4f20, 0x1849b5, 0x1d71840, 0xc42067c100, 0xc2483b8, 0xed3108808, 0x0, 0x30003, 0xc423158fae) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:149 +0x114 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc422bb6e60, 0x1849b5, 0xc4224a3740, 0xf56b0f, 0x1, 0x0, 0x0, 0x0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc422bb6e60, 0x1849b5, 0xc4224a39e0) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26091]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:29:35 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:29:35 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:29:35 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: ==> Starting Nomad agent... Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: ==> Nomad agent configuration: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: Client: false Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: Log Level: INFO Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: Region: amrv01 (DC: test) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: Server: true Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: Version: 0.8.4 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:35 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:35 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:35 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:35 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:35.967564 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:35.967663 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:29:35 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:35 [INFO] serf: Attempting re-join to previously known node: ip-172-29-20-70.atf01: 172.29.20.70:4648 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.084376 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.146499 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.146536 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.146550 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.146946 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.146959 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.146971 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147010 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147022 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147033 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147044 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147056 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147131 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147145 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147156 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147167 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [WARN] memberlist: Refuting an alive message Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147178 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147189 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147200 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36 [INFO] serf: Re-joined to previously known node: ip-172-29-20-70.atf01: 172.29.20.70:4648 Aug 23 15:29:36 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:36.147277 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:37 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:37 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:38 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:38.552619 [ERR] worker: failed to ack evaluation '51ef8043-e99c-9686-46ce-8ae77f7367c9': rpc error: EOF Aug 23 15:29:38 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:38.582795 [ERR] worker: failed to update evaluation : rpc error: stream closed Aug 23 15:29:38 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:38.582848 [ERR] worker: failed to nack evaluation '51ef8043-e99c-9686-46ce-8ae77f7367c9': rpc error: stream closed Aug 23 15:29:38 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:38.583889 [ERR] worker: failed to update evaluation : rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:38 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:38.583892 [ERR] worker: failed to update evaluation : rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:38 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:38.584895 [ERR] worker: failed to nack evaluation '51ef8043-e99c-9686-46ce-8ae77f7367c9': rpc error: failed to get conn: dial tcp 172.24.10.89:4647: connect: connection refused Aug 23 15:29:38 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:38.584967 [ERR] worker: failed to nack evaluation '51ef8043-e99c-9686-46ce-8ae77f7367c9': rpc error: failed to get conn: rpc error: lead thread didn't get connection Aug 23 15:29:40 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:40 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:29:40 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:40 [INFO] raft: Duplicate RequestVote for same term: 75 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42.066891 [INFO] server.nomad: successfully contacted 20 Nomad Servers Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42.263252 [ERR] http: Request /v1/agent/health?type=server, error: {"server":{"ok":false,"message":"rpc error: stream closed"}} Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42 [WARN] raft: Rejecting vote request from 172.24.0.121:4647 since we have a leader: 172.24.10.89:4647 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42 [WARN] raft: Heartbeat timeout from "172.24.10.89:4647" reached, starting election Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 76 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42 [INFO] raft: Election won. Tally: 2 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42.565625 [INFO] nomad: cluster leadership acquired Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: 2018/08/23 15:29:42 [INFO] raft: pipelining replication to peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: fatal error: concurrent map read and map write Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 322 [running]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc4231acd20 sp=0xc4231acd00 pc=0x42d161 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: runtime.mapaccess2(0x1844be0, 0xc4203755c0, 0xc4231acde0, 0xc42056f358, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc4231acd68 sp=0xc4231acd20 pc=0x409c75 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc4202bf8a0, 0x1d6dd40, 0xc42056f358, 0x0, 0xed3109fb6) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:156 +0xdc fp=0xc4231ace50 sp=0xc4231acd68 pc=0xf215ac Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4204a4000, 0x1d71840, 0xc423218c00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc4231acfc8 sp=0xc4231ace50 pc=0xf56bb7 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: runtime.goexit() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc4231acfd0 sp=0xc4231acfc8 pc=0x45c711 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 1 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42030f420, 0xc420490280) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42030f420, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc420312280, 0xc420312280, 0xc42016f860, 0x37) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204e7f68) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: main.main() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 5 [syscall]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: os/signal.signal_recv(0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: os/signal.loop() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by os/signal.init.0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 69 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4201fd080, 0x1d71840, 0xc420550000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 150 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420d1c0d0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 27 [sleep]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: time.Sleep(0x3b9aca00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420516000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 153 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4204863c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 26 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc4203e78c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 183 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420232420) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 8 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420572140, 0x1c636a0, 0xc42025ebb0, 0x1d71840, 0xc4203f6f00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc42025ebb0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 42 [select, locked to thread]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: runtime.selectgo(0xc4204b8f50, 0xc420090660) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: runtime.ensureSigM.func1() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: runtime.goexit() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 28 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc4201da3c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 29 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4201f4540) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 149 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420ab5e00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 31 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc42055a000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc420375a10, 0xc42031a270) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 94 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204c43c0, 0xc420aea000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc420d15b00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc420d15b00, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420dcd650, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420704820) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d15b60, 0x16e4b60, 0xc420704880) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d15b60, 0x16e4b60, 0xc420704880, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4211ae700, 0x16e4b60, 0xc420704880, 0xc420704880, 0xc4201ffce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4211ae700, 0xc420704880, 0x2d, 0xc420853180) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequestHeader(0xc4201ffcc0, 0x1d72340, 0xc4211ae700, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9b30, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequest(0xc4201ffcc0, 0x1d72340, 0xc4211ae700, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc4201ffcc0, 0x1d72340, 0xc4211ae700, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc4204c43c0, 0xc4201ffcc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 246 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422898210, 0xc42125cd10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422898210, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc422b42260, 0xc42101fc20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc422b42260, 0xc42101fc20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 44 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613ff00, 0x72, 0xc4205a19a8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420564198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420564198, 0xc4205a6000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc420564180, 0xc4205a6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc420564180, 0xc4205a6000, 0x1000, 0x1000, 0x458600, 0xc4202e6780, 0x4) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc42000ea00, 0xc4205a6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/http.(*persistConn).Read(0xc420574120, 0xc4205a6000, 0x1000, 0x1000, 0xc4205a1b98, 0x405675, 0xc4200907e0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc4200875c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Peek(0xc4200875c0, 0x1, 0x0, 0x0, 0x1, 0xc420de22a0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/http.(*persistConn).readLoop(0xc420574120) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by net/http.(*Transport).dialConn Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 45 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/http.(*persistConn).writeLoop(0xc420574120) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by net/http.(*Transport).dialConn Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 86 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205160b0, 0xc420066d10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205160b0, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc420202080, 0xc4201ffc20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc420202080, 0xc4201ffc20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 88 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205160b0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 148 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc420390000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420390000, 0xc4205c0620) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 264 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613f470, 0x72, 0xc421263ac8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc4228f2218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc4228f2218, 0xc422ad2000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc4228f2200, 0xc422ad2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc4228f2200, 0xc422ad2000, 0x1000, 0x1000, 0xc4203f6f60, 0xc421263be0, 0x7b9b0e) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc420202ef0, 0xc422ad2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc421eca060) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc421eca060, 0x3, 0xc420044a70, 0xc420044a00) Aug 23 15:29:42 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleCommand(0xc42025ebb0, 0xc421eca060, 0xc421eca0c0, 0xc422bd6070, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:515 +0x43 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleConn(0xc42025ebb0, 0x1d71840, 0xc4203f6f00, 0x1d7d320, 0xc420202ef0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:499 +0x23d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:477 +0x1ce Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 93 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204c4300, 0xc420ad2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc420d158c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc420d158c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420dcd470, 0x42c30b) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420704740) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d15920, 0x16e4b60, 0xc4207047a0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d15920, 0x16e4b60, 0xc4207047a0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4211ae540, 0x16e4b60, 0xc4207047a0, 0xc4207047a0, 0xc4201ffce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4211ae540, 0xc4207047a0, 0x2d, 0xc420b10be0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequestHeader(0xc4201ffcc0, 0x1d72340, 0xc4211ae540, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequest(0xc4201ffcc0, 0x1d72340, 0xc4211ae540, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc4201ffcc0, 0x1d72340, 0xc4211ae540, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc4204c4300, 0xc4201ffcc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 89 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205160b0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 240 [select]: Aug 23 15:29:42 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42098c180, 0xc422fcf000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc4213d9500) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc4213d9500, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc423277ad0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc423238200) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4213d9560, 0x16e4b60, 0xc423238260) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4213d9560, 0x16e4b60, 0xc423238260, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc423278600, 0x16e4b60, 0xc423238260, 0xc423238260, 0xc4225ce2f8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc423278600, 0xc423238260, 0x2d, 0xc421766740) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequestHeader(0xc4225ce2d0, 0x1d72340, 0xc423278600, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9b30, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequest(0xc4225ce2d0, 0x1d72340, 0xc423278600, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc4225ce2d0, 0x1d72340, 0xc423278600, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc42098c180, 0xc4225ce2d0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 87 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613fa20, 0x72, 0xc420fc6ce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420445a18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420445a18, 0xc420c5c000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc420445a00, 0xc420c5c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc420445a00, 0xc420c5c000, 0x1000, 0x1000, 0xc420fc6dd8, 0x0, 0xc420fc6e08) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc420202080, 0xc420c5c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc420902300, 0xc420b26160, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc420902300, 0xc420b26160, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc420902300, 0xc420b26160, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205160b0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205160b0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 237 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab1298add0, 0x72, 0xc420fc3ce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc4210f4118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc4210f4118, 0xc422fce000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc4210f4100, 0xc422fce000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc4210f4100, 0xc422fce000, 0x1000, 0x1000, 0xc4204b85d8, 0x1, 0xc4204b8608) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc422b42010, 0xc422fce000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc4205d65a0, 0xc4227ec930, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc4205d65a0, 0xc4227ec930, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc4205d65a0, 0xc4227ec930, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4200be2c0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4200be2c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 147 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc420390000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420390000, 0xc4205c0610) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 236 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4200be2c0, 0xc421132d10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4200be2c0, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc422b42010, 0xc4225ce230) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc422b42010, 0xc4225ce230) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 151 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613fd60, 0x72, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420e0e018, 0x72, 0xc4203e7b00, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420e0e018, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Accept(0xc420e0e000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).accept(0xc420e0e000, 0xc422b42270, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*TCPListener).accept(0xc42056e068, 0x458e40, 0xc42002b790, 0xc42002b798) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*TCPListener).AcceptTCP(0xc42056e068, 0x1c63658, 0xc4204863c0, 0xc422b42270) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4204863c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 124 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42025ee70) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 146 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc420390000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc420390000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc420390000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420390000, 0xc4205c0600) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 152 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613fc90, 0x72, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420e0e298, 0x72, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420e0e298, 0xc4236bc000, 0x10000, 0x10000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).ReadFrom(0xc420e0e280, 0xc4236bc000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).readFrom(0xc420e0e280, 0xc4236bc000, 0x10000, 0x10000, 0x28, 0x28, 0x1d62b40, 0x412539, 0xc4230cbe40) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*UDPConn).readFrom(0xc42056e088, 0xc4236bc000, 0x10000, 0x10000, 0x101, 0xc4230cbe18, 0xc4230cbe90, 0x443cf7) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*UDPConn).ReadFrom(0xc42056e088, 0xc4236bc000, 0x10000, 0x10000, 0x10000, 0x10000, 0x6fc4, 0x19646e07f, 0x2b96de0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4204863c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 154 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4204863c0, 0x12a05f200, 0xc420ba5ec0, 0xc420edfc20, 0xc4205c0be0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 155 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4204863c0, 0xc420edfc20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 156 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4204863c0, 0x1dcd6500, 0xc420ba5f20, 0xc420edfc20, 0xc4205c0c00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 157 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4205b0d80) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 158 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4205b0d80) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 159 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4205b0d80, 0x1bdd09a, 0x6, 0xc42086a2d0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 160 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4205b0d80, 0x1bdba6c, 0x5, 0xc42086a330) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 161 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4205b0d80, 0x1bdbc6a, 0x5, 0xc42086a3c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 239 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4200be2c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 163 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420ba5f80, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ba5f80, 0x1dcd6500, 0x24, 0xc42244ad20, 0x24, 0x1, 0x1c66000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ba5f80) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 164 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420a00000, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420a00000, 0x1dcd6500, 0x24, 0xc42244a540, 0x24, 0xc42064ac00, 0xc420c4fe00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420a00000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 165 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420a00060, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420a00060, 0x1dcd6500, 0x24, 0xc4223de720, 0x24, 0xc4222ece00, 0x1c66000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420a00060) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 166 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420a000c0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420a000c0, 0x1dcd6500, 0x24, 0xc4223de450, 0x24, 0xc4222ecf00, 0x2d633438332d3400) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420a000c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 167 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420cf5170, 0xc4200e0280) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 168 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc4200e0280) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 169 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc4200e0280) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 170 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613fe30, 0x72, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420564418, 0x72, 0xc4203e7900, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420564418, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Accept(0xc420564400, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).accept(0xc420564400, 0x1, 0x1, 0xc420060f20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*TCPListener).accept(0xc42056e0b0, 0x406b4a, 0xc420ecf680, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*TCPListener).Accept(0xc42056e0b0, 0xc420ecf680, 0x0, 0x3f800000, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 171 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4204a4000, 0x3b9aca00, 0xc420336480) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 172 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc4201ff090, 0x3b9aca00, 0xc420336480) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 173 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc420524120, 0x3b9aca00, 0xc420336480) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 174 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc42055a000, 0x3b9aca00, 0xc420336480) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 175 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc4200e0280) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 176 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613faf0, 0x72, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420e0ef98, 0x72, 0xc420e4e400, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420e0ef98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Accept(0xc420e0ef80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).accept(0xc420e0ef80, 0x402037, 0xc4217d8260, 0x1805ec0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*TCPListener).accept(0xc42056e540, 0x458e40, 0xc421261e90, 0xc421261e98) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*TCPListener).Accept(0xc42056e540, 0x1c65af8, 0xc4217d81e0, 0x1d71900, 0xc420d3ea20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/http.(*Server).Serve(0xc4207c81a0, 0x1d6fc00, 0xc42056e540, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/http.Serve(0x1d6fc00, 0xc42056e540, 0x1d5df00, 0xc420d3e5d0, 0x1c632f0, 0xc420b74d80) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420eba500, 0x1d6fc00, 0xc42056e540, 0xc4205c1180, 0xc420cf5440) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 177 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420e0f000, 0xc4205c1190, 0xc420eb9320) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 178 [sleep]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: time.Sleep(0x4add3b872) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc42030f420, 0xc420e0f000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 95 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204c4480, 0xc42094c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc420d15d40) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc420d15d40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420dcdad0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207049c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d15da0, 0x16e4b60, 0xc420704a20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d15da0, 0x16e4b60, 0xc420704a20, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4211ae7c0, 0x16e4b60, 0xc420704a20, 0xc420704a20, 0xc4201ffce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4211ae7c0, 0xc420704a20, 0x2d, 0xc4209ac300) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequestHeader(0xc4201ffcc0, 0x1d72340, 0xc4211ae7c0, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequest(0xc4201ffcc0, 0x1d72340, 0xc4211ae7c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc4201ffcc0, 0x1d72340, 0xc4211ae7c0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc4204c4480, 0xc4201ffcc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 96 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204c46c0, 0xc420994000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc420d15f20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc420d15f20, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420dcdc80, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420704a80) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d15f80, 0x16e4b60, 0xc420704ae0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d15f80, 0x16e4b60, 0xc420704ae0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4211ae880, 0x16e4b60, 0xc420704ae0, 0xc420704ae0, 0xc4201ffce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4211ae880, 0xc420704ae0, 0x2d, 0xc421112360) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequestHeader(0xc4201ffcc0, 0x1d72340, 0xc4211ae880, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9b30, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequest(0xc4201ffcc0, 0x1d72340, 0xc4211ae880, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc4201ffcc0, 0x1d72340, 0xc4211ae880, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc4204c46c0, 0xc4201ffcc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 285 [runnable]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: type..hash.github.com/hashicorp/nomad/nomad/structs.NamespacedID(0xc421135ac8, 0xde1b00d7, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: :1 +0x51 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*EvalBroker).enqueueLocked(0xc4204a4000, 0xc420689340, 0xc4201bc587, 0x5) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:279 +0xb8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4204a4000, 0xc420689340, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:245 +0x2d6 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4204a4000, 0xc420689340) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc4200e0280, 0xc42050b701, 0xc42050b740) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc4200e0280, 0xc4232ca6c0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc4200e0280, 0xc4232ca6c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc4209fe0e0, 0xc4200e0280, 0xc4232ca6c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 194 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4204c4840, 0xc420d7e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc420d15320) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc420d15320, 0x101000000000016, 0x0, 0x2d) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420dcc2d0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420704400) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d15380, 0x16e4b60, 0xc420704480) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d15380, 0x16e4b60, 0xc420704480, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4211ae0c0, 0x16e4b60, 0xc420704480, 0xc420704480, 0xc4201ffce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4211ae0c0, 0xc420704480, 0x2d, 0xc421b4b580) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequestHeader(0xc4201ffcc0, 0x1d72340, 0xc4211ae0c0, 0x0, 0x0, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequest(0xc4201ffcc0, 0x1d72340, 0xc4211ae0c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc4201ffcc0, 0x1d72340, 0xc4211ae0c0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc4204c4840, 0xc4201ffcc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 274 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613fbc0, 0x72, 0xc42115fce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc422de6c98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc422de6c98, 0xc422ad3000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc422de6c80, 0xc422ad3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc422de6c80, 0xc422ad3000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc422b42260, 0xc422ad3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc4210035c0, 0xc422c2ab00, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc4210035c0, 0xc422c2ab00, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc4210035c0, 0xc422c2ab00, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422898210, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422898210) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 182 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420232420) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 123 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42025ee70) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 199 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613f880, 0x72, 0xc420fe69a8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420d4f198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420d4f198, 0xc420f10000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc420d4f180, 0xc420f10000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc420d4f180, 0xc420f10000, 0x1000, 0x1000, 0x458600, 0xc4202e7680, 0x4) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc420203260, 0xc420f10000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/http.(*persistConn).Read(0xc420d6a000, 0xc420f10000, 0x1000, 0x1000, 0xc420fe6b98, 0x405675, 0xc420ea65a0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc420c44900) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Peek(0xc420c44900, 0x1, 0x0, 0x0, 0x1, 0xc420ea72c0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/http.(*persistConn).readLoop(0xc420d6a000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by net/http.(*Transport).dialConn Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 200 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/http.(*persistConn).writeLoop(0xc420d6a000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by net/http.(*Transport).dialConn Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 122 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613f7b0, 0x72, 0xc420fe2ce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420d4f318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420d4f318, 0xc420e66000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc420d4f300, 0xc420e66000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc420d4f300, 0xc420e66000, 0x1000, 0x1000, 0xc421258dd8, 0x1, 0xc421258e08) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc420203298, 0xc420e66000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc420b913e0, 0xc420716c80, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc420b913e0, 0xc420716c80, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc420b913e0, 0xc420716c80, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42025ee70, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42025ee70) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 202 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42025ee70, 0xc4205a5d10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42025ee70, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc420203298, 0xc420c046e0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc420203298, 0xc420c046e0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 125 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4201da6c0, 0xc420e67000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc420de5740) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc420de5740, 0x101000000000016, 0x0, 0x2d) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42127ba10, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420732d80) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420de57a0, 0x16e4b60, 0xc420732de0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420de57a0, 0x16e4b60, 0xc420732de0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420dd5bc0, 0x16e4b60, 0xc420732de0, 0xc420732de0, 0xc420608708) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420dd5bc0, 0xc420732de0, 0x2d, 0xc42100b1c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequestHeader(0xc4206086e0, 0x1d72340, 0xc420dd5bc0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a9b30, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequest(0xc4206086e0, 0x1d72340, 0xc420dd5bc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc4206086e0, 0x1d72340, 0xc420dd5bc0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc4201da6c0, 0xc4206086e0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 181 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613f950, 0x72, 0xc420fe0ce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc4208ee098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc4208ee098, 0xc42095c000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc4208ee080, 0xc42095c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc4208ee080, 0xc42095c000, 0x1000, 0x1000, 0xc420055dd8, 0x1, 0xc420055e08) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc42020c1e8, 0xc42095c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc420a01020, 0xc4206b4e90, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc420a01020, 0xc4206b4e90, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc420a01020, 0xc4206b4e90, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420232420, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420232420) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 134 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420232420, 0xc420fe3d10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420232420, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42020c1e8, 0xc4209b8e60) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42020c1e8, 0xc4209b8e60) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 184 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420912600, 0xc42095d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc420d0f380) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc420d0f380, 0x1c66260, 0xffffffffffffffff, 0xc42116fab0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420c3cfc0, 0x1) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206b63c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d0f3e0, 0x16e4b60, 0xc4206b6420) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d0f3e0, 0x16e4b60, 0xc4206b6420, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4207be5c0, 0x16e4b60, 0xc4206b6420, 0xc4206b6420, 0xc42105a988) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4207be5c0, 0xc4206b6420, 0x2d, 0xc4211131a0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequestHeader(0xc42105a960, 0x1d72340, 0xc4207be5c0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9b30, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequest(0xc42105a960, 0x1d72340, 0xc4207be5c0, 0x3, 0x3, 0xc42116ff50, 0xc42116fe18, 0xb8, 0xc, 0xc420040070, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc42105a960, 0x1d72340, 0xc4207be5c0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc420912600, 0xc42105a960) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 212 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42025edc0, 0xc420fc1d10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42025edc0, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42056e000, 0xc421250000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42056e000, 0xc421250000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 275 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422898210) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 213 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613f3a0, 0x72, 0xc420fc4ce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc42124a018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc42124a018, 0xc420dde000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc42124a000, 0xc420dde000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc42124a000, 0xc420dde000, 0x1000, 0x1000, 0xc4200555d8, 0x1, 0xc420055608) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc42056e000, 0xc420dde000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc4213d8060, 0xc4207080c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc4213d8060, 0xc4207080c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc4213d8060, 0xc4207080c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42025edc0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42025edc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 238 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4200be2c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 139 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4227f4e70, 0xc421161d10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4227f4e70, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42056fdc8, 0xc420573e50) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42056fdc8, 0xc420573e50) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 224 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613f130, 0x72, 0xc421136ce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc4208eff98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc4208eff98, 0xc421db1000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc4208eff80, 0xc421db1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc4208eff80, 0xc421db1000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc42056fdc8, 0xc421db1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc4219a7500, 0xc422883eb0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc4219a7500, 0xc422883eb0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc4219a7500, 0xc422883eb0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4227f4e70, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4227f4e70) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 214 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42025edc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 215 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42025edc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 216 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420912240, 0xc421410000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc4214084e0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc4214084e0, 0x1c66260, 0x0, 0xc421173ab0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42124d500, 0x1) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4212520c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421408540, 0x16e4b60, 0xc421252120) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421408540, 0x16e4b60, 0xc421252120, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4213e24c0, 0x16e4b60, 0xc421252120, 0xc421252120, 0xc4212500c8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4213e24c0, 0xc421252120, 0x2d, 0xc420cd36a0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequestHeader(0xc4212500a0, 0x1d72340, 0xc4213e24c0, 0x0, 0x2, 0x3, 0x100000000000003, 0x0, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequest(0xc4212500a0, 0x1d72340, 0xc4213e24c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc4212500a0, 0x1d72340, 0xc4213e24c0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc420912240, 0xc4212500a0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 226 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42025efd0, 0xc421262d10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42025efd0, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42056e5f0, 0xc42133dea0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42056e5f0, 0xc42133dea0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 227 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613f2d0, 0x72, 0xc421162ce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420d10b18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420d10b18, 0xc42122a000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc420d10b00, 0xc42122a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc420d10b00, 0xc42122a000, 0x1000, 0x1000, 0xaa, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc42056e5f0, 0xc42122a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc421344fc0, 0xc420760000, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc421344fc0, 0xc420760000, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc421344fc0, 0xc420760000, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42025efd0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42025efd0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 228 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42025efd0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 229 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42025efd0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 230 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420912180, 0xc42122b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc420a008a0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc420a008a0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420c514a0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206110a0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420a00900, 0x16e4b60, 0xc420611100) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420a00900, 0x16e4b60, 0xc420611100, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4210e9e80, 0x16e4b60, 0xc420611100, 0xc420611100, 0xc42133df68) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4210e9e80, 0xc420611100, 0x2d, 0xc420755580) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequestHeader(0xc42133df40, 0x1d72340, 0xc4210e9e80, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a9b30, 0x8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).readRequest(0xc42133df40, 0x1d72340, 0xc4210e9e80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc42133df40, 0x1d72340, 0xc4210e9e80, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc420912180, 0xc42133df40) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 218 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420233550, 0xc42115ed10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420233550, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc420203250, 0xc421087360) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc420203250, 0xc421087360) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 219 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613f200, 0x72, 0xc421139ce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420e0e718, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420e0e718, 0xc421b56000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc420e0e700, 0xc421b56000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc420e0e700, 0xc421b56000, 0x1000, 0x1000, 0xc42004edd8, 0x1, 0xc42004ee08) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc420203250, 0xc421b56000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc420a01680, 0xc421c14340, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc420a01680, 0xc421c14340, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc420a01680, 0xc421c14340, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420233550, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420233550) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 220 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420233550) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 221 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420233550) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 222 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc421003e60, 0xc42126b700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc421a43b90, 0x1d718c0, 0xc421003e60, 0x0, 0xc421003e00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e0280, 0xc42126b950, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4210037a0, 0xc420564900, 0xc421a439e0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: reflect.Value.call(0xc4205b4960, 0xc420203578, 0x13, 0x1bda9a0, 0x4, 0xc421263d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: reflect.Value.Call(0xc4205b4960, 0xc420203578, 0x13, 0xc421263d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*service).call(0xc4218a6e80, 0xc421087400, 0xc421845fb0, 0x0, 0xc421511c80, 0xc42163a1e0, 0x19758e0, 0xc420564900, 0x16, 0x169e020, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc421087400, 0x1d72340, 0xc4218a6ec0, 0x0, 0xc4204c4c00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc4204c4c00, 0xc421087400) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 292 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422185ef0, 0xc423270d10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422185ef0, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42020c950, 0xc4222f40a0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42020c950, 0xc4222f40a0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 225 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4227f4e70) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 242 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4227f4e70) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 243 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420fb3f20, 0xc4231bb700, 0x20, 0x20, 0x169d4e0, 0xc421599f00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4221e64e0, 0x1d718c0, 0xc420fb3f20, 0x0, 0xc420fb3f00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e0280, 0xc4231bb950, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420fb37a0, 0xc42209fe00, 0xc4221e6300, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: reflect.Value.call(0xc4205b4960, 0xc42000f438, 0x13, 0x1bda9a0, 0x4, 0xc421163d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: reflect.Value.Call(0xc4205b4960, 0xc42000f438, 0x13, 0xc421163d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*service).call(0xc4230b7500, 0xc422822a00, 0xc4221b2e38, 0x0, 0xc42209f980, 0xc42311d080, 0x19758e0, 0xc42209fe00, 0x16, 0x169e020, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc422822a00, 0x1d72340, 0xc4230b7540, 0x0, 0xc420912e40) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc420912e40, 0xc422822a00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 276 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422898210) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 277 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4232e66c0, 0xc4231b5700, 0x20, 0x20, 0x169d4e0, 0xc421599f00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4232a8600, 0x1d718c0, 0xc4232e66c0, 0x0, 0xc4232e6600) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e0280, 0xc4231b5950, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4205d7380, 0xc4208eef00, 0xc4232a8510, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: reflect.Value.call(0xc4205b4960, 0xc42000fc68, 0x13, 0x1bda9a0, 0x4, 0xc4231b5d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: reflect.Value.Call(0xc4205b4960, 0xc42000fc68, 0x13, 0xc4231b5d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*service).call(0xc42274bf00, 0xc422406370, 0xc421b84cc8, 0x0, 0xc421dc4300, 0xc420adba80, 0x19758e0, 0xc4208eef00, 0x16, 0x169e020, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc422406370, 0x1d72340, 0xc42274bf80, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc4204c5ec0, 0xc422406370) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 287 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420a70780) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 286 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc420390000, 0xc4209126c0, 0xc4232ca720) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420390000, 0xc42066ac00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 288 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420a70780) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 310 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc420390000, 0xc420912540, 0xc423314840) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420390000, 0xc423283c20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 268 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42025f3f0, 0xc421137d10, 0x30, 0xf9866a) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42025f3f0, 0x1c61988, 0xc4200e0280, 0x1d71840, 0xc420dffdc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42056ed10, 0xc420519180) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d320, 0xc42056ed10, 0xc420519180) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 269 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab1298aea0, 0x72, 0xc420063ce8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc42263cb18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc42263cb18, 0xc42322b000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc42263cb00, 0xc42322b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc42263cb00, 0xc42322b000, 0x1000, 0x1000, 0xc4212565d8, 0x1, 0xc421256608) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc42056ed10, 0xc42322b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc421002ae0, 0xc422b80bd0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc421002ae0, 0xc422b80bd0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc421002ae0, 0xc422b80bd0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42025f3f0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42025f3f0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 270 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42025f3f0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 271 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42025f3f0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 272 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420d14180, 0xc421265700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc42328c6c0, 0x1d718c0, 0xc420d14180, 0x0, 0xc420d14100) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc4200e0280, 0xc421265950, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420b91b60, 0xc420e0ed00, 0xc42328c5d0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: reflect.Value.call(0xc4205b4960, 0xc42056f018, 0x13, 0x1bda9a0, 0x4, 0xc42059ed48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: reflect.Value.Call(0xc4205b4960, 0xc42056f018, 0x13, 0xc42059ed48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*service).call(0xc4232a0780, 0xc420519220, 0xc421b85fb0, 0x0, 0xc420e0e180, 0xc421571040, 0x19758e0, 0xc420e0ed00, 0x16, 0x169e020, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net/rpc.(*Server).ServeRequest(0xc420519220, 0x1d72340, 0xc4232a07c0, 0x0, 0xc42098c0c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc4200e0280, 0x1d71840, 0xc420dffdc0, 0x1d7d0e0, 0xc42098c0c0, 0xc420519220) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 255 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc420390000, 0xc420912540, 0x1d6eb80, 0xc4232a1100, 0xc4232f6900, 0xc4232f6a20) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420390000, 0xc4232a1140) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 253 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613f6e0, 0x72, 0xc421138858) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc4208ef998, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc4208ef998, 0xc422530000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc4208ef980, 0xc422530000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc4208ef980, 0xc422530000, 0x1000, 0x1000, 0xc420040000, 0xc4211ae098, 0xc4211ae0a0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc420202f28, 0xc422530000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).fill(0xc420d14900) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).ReadByte(0xc420d14900, 0x72, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4231d4960, 0xc4208ef998) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421571900) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d14960, 0x16e76e0, 0xc420695b80) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d14960, 0x16e76e0, 0xc420695b80, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.decodeResponse(0xc4232a0d00, 0x182ba60, 0xc4225b1a00, 0xc420de53e0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:608 +0x5e Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).genericRPC(0xc42025ebb0, 0xc42038c1e0, 0x24, 0xc420358240, 0x11, 0x21b8216420de5300, 0x182ba00, 0xc420de53e0, 0x182ba60, 0xc4225b1a00, ...) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:406 +0xf9 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).AppendEntries(0xc42025ebb0, 0xc42038c1e0, 0x24, 0xc420358240, 0x11, 0xc420de53e0, 0xc4225b1a00, 0x1, 0x3) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:379 +0x97 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc420390000, 0xc4209126c0, 0x18640e, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:186 +0x189 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc420390000, 0xc4209126c0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:132 +0x3f2 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420390000, 0xc421571bc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 252 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc420390000, 0xc420912540, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc420390000, 0xc420912540) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420390000, 0xc421571ba0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 254 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4232a1100) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 293 [IO wait]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.runtime_pollWait(0x7fab3613f060, 0x72, 0xc42059ece8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).wait(0xc420134598, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*pollDesc).waitRead(0xc420134598, 0xc4230d6000, 0x1000, 0x1000) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: internal/poll.(*FD).Read(0xc420134580, 0xc4230d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*netFD).Read(0xc420134580, 0xc4230d6000, 0x1000, 0x1000, 0xc42059edd8, 0x0, 0xc42059ee08) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: net.(*conn).Read(0xc42020c950, 0xc4230d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: bufio.(*Reader).Read(0xc420c44240, 0xc422a8af80, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadAtLeast(0x1d59aa0, 0xc420c44240, 0xc422a8af80, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: io.ReadFull(0x1d59aa0, 0xc420c44240, 0xc422a8af80, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422185ef0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422185ef0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 294 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422185ef0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 295 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422185ef0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 297 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422c53e00, 0xc4232e1980) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 289 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc4201ff090, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc4200e0280) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 323 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc420524120) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 324 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc420524120) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 325 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc42328db90) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 326 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc423218c40, 0xc422d40ce8, 0x20, 0x20, 0x18, 0xc423283ec0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422feecf0, 0x1d71840, 0xc423218c40, 0xc423283ec0, 0x184987) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42050b740, 0xc420c61ed0, 0x184987, 0x1d71840, 0xc423218c40, 0xc420e0e5e0, 0xc42328db60, 0xc420718810, 0x24, 0x2bba400) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420e0e580, 0x1d71840, 0xc423218c40, 0x184987, 0xc423245640, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420e0e580, 0x1d71840, 0xc423218c40) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 327 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc423218d00, 0xc42325f770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422fef050, 0x1d71840, 0xc423218d00, 0x0, 0x1849b5) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42050b740, 0xc42325f960, 0x1849b5, 0x1d71840, 0xc423218d00, 0x552bde, 0xc42325f950, 0xc42325f970, 0xf33ba5, 0xc420489cc8) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc420489c70, 0x1d71840, 0xc423218d00, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc420489c70) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 328 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc423218cc0, 0xc422c17a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4231d5b00, 0x1d71840, 0xc423218cc0, 0xc4231d5bf0, 0x1833a4) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42050b740, 0xc422c17c68, 0x1833a4, 0x1d71840, 0xc423218cc0, 0x552bde, 0xc4201fee10, 0x2, 0xc420619340, 0x40) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc423218d40, 0x1833a4, 0x3b, 0xc4231b3de8, 0x1, 0x1) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc423218d40) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 329 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc423218d80) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 330 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4204865a0, 0x1d71840, 0xc423218cc0) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 298 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422c53e00, 0xc4232e1b00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 311 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc4217d8280) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: goroutine 313 [select]: Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc423245700, 0xc421163d28, 0x20, 0x20, 0xc4202e4800, 0x1bd9a98) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422fef170, 0x1d71840, 0xc423245700, 0xc422950300, 0x1849b5) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42050b740, 0xc4230c7f20, 0x1849b5, 0x1d71840, 0xc423245700, 0xf28cc9, 0xc420e0e580, 0x1d71840, 0xc423218c40, 0x1) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc4217d8280, 0x1849b5, 0xb8, 0xc42088f980, 0xbc, 0xc4218fbcc0, 0xc0, 0xc42088fa00) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc4217d8280, 0x1849b5, 0xc4202f4c60) Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:29:42 ip-172-24-20-236 nomad.sh[26113]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:29:42 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:29:42 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:29:42 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: ==> Starting Nomad agent... Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: ==> Nomad agent configuration: Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: Client: false Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: Log Level: INFO Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: Region: amrv01 (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: Server: true Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: Version: 0.8.4 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.237145 [INFO] nomad: starting 4 scheduling worker(s) for [batch system service _core] Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: Attempting re-join to previously known node: vol-cl-control-03.global: 172.16.9.35:4648 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.237396 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.305834 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546269 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546292 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546311 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546323 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546335 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546347 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546637 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546660 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546672 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546684 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.546695 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.547008 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.547028 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.547041 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.547053 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.547064 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [WARN] memberlist: Refuting an alive message Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.547313 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43 [INFO] serf: Re-joined to previously known node: vol-cl-control-03.global: 172.16.9.35:4648 Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.547338 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:29:43 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:43.547352 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 77 Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [INFO] raft: Election won. Tally: 2 Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44.419959 [INFO] nomad: cluster leadership acquired Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [INFO] raft: pipelining replication to peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [WARN] raft: AppendEntries to {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} rejected, sending older logs (next: 1598479) Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:44 ip-172-24-20-236 nomad.sh[26133]: 2018/08/23 15:29:44 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: fatal error: concurrent map writes Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 229 [running]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc4204c4508 sp=0xc4204c44e8 pc=0x42d161 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: runtime.mapdelete(0x1844be0, 0xc42052a360, 0xc4204c4620) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/hashmap.go:630 +0x311 fp=0xc4204c4568 sp=0xc4204c4508 pc=0x40a761 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc4203c0880, 0x1d6dd40, 0xc42000efa8, 0x0, 0xed3109fb9) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:158 +0x17c fp=0xc4204c4650 sp=0xc4204c4568 pc=0xf2164c Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc4203fd2c0, 0x1d71840, 0xc42242bd80) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc4204c47c8 sp=0xc4204c4650 pc=0xf56bb7 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: runtime.goexit() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc4204c47d0 sp=0xc4204c47c8 pc=0x45c711 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 1 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42031bc00, 0xc4204f32c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42031bc00, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc420241180, 0xc420241180, 0xc420203d80, 0x37) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204e7f68) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: main.main() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 5 [syscall]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: os/signal.signal_recv(0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: os/signal.loop() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by os/signal.init.0 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 118 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc4203cc1a0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 148 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b4000, 0xc420cf6000, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b4000, 0xc420cf6000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4206ee480) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 15 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc42001d640, 0x1d71840, 0xc420402ec0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 28 [select, locked to thread]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: runtime.selectgo(0xc420050f50, 0xc4204f4840) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: runtime.ensureSigM.func1() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: runtime.goexit() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 29 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc42001d580) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 30 [sleep]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: time.Sleep(0x3b9aca00) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc42023f760) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 31 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc4202386c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 32 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4200878c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 50 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc420021790) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0xc4202b1f00, 0xc4202ace80) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc42052a7b0, 0xc4203989e0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 164 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420238a80, 0xc420a6d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).fill(0xc420b05ec0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).ReadByte(0xc420b05ec0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420806960, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42070b1c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420b05f20, 0x16e4b60, 0xc42070b220) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420b05f20, 0x16e4b60, 0xc42070b220, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e90080, 0x16e4b60, 0xc42070b220, 0xc42070b220, 0xc42053e5c8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e90080, 0xc42070b220, 0x2d, 0xc42306a4c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequestHeader(0xc42053e5a0, 0x1d72340, 0xc420e90080, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9d10, 0x8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequest(0xc42053e5a0, 0x1d72340, 0xc420e90080, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).ServeRequest(0xc42053e5a0, 0x1d72340, 0xc420e90080, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d0e0, 0xc420238a80, 0xc42053e5a0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 149 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4200b4000, 0xc420cf60c0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4200b4000, 0xc420cf60c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4206ee4a0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 117 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420885ec0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 57 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4203c3360, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc42023e160) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 55 [IO wait]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.runtime_pollWait(0x7f006138df00, 0x72, 0xc4205479a8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).wait(0xc420134e18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:45 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).waitRead(0xc420134e18, 0xc42058e000, 0x1000, 0x1000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*FD).Read(0xc420134e00, 0xc42058e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*netFD).Read(0xc420134e00, 0xc42058e000, 0x1000, 0x1000, 0x458600, 0xc4202f3680, 0x4) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*conn).Read(0xc420204be8, 0xc42058e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/http.(*persistConn).Read(0xc420550120, 0xc42058e000, 0x1000, 0x1000, 0xc420547b98, 0x405675, 0xc420592060) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).fill(0xc42054ecc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).Peek(0xc42054ecc0, 0x1, 0x0, 0x0, 0x1, 0xc420f046c0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/http.(*persistConn).readLoop(0xc420550120) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by net/http.(*Transport).dialConn Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 206 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42023e420, 0xc421123d10, 0x30, 0xf9866a) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42023e420, 0x1c61988, 0xc420010280, 0x1d71840, 0xc420a48240) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d320, 0xc42021a020, 0xc42138e230) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d320, 0xc42021a020, 0xc42138e230) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:45 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 56 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/http.(*persistConn).writeLoop(0xc420550120) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by net/http.(*Transport).dialConn Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 114 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4200b4000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4200b4000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4200b4000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420f42620) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 115 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4200b4000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:29:45 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420f42630) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 93 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42023e210, 0xc420fe2d10, 0x30, 0xf9866a) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42023e210, 0x1c61988, 0xc420010280, 0x1d71840, 0xc420a48240) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d320, 0xc42059a000, 0xc42053e4b0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d320, 0xc42059a000, 0xc42053e4b0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 116 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4200b4000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420f42640) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 195 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420310180, 0xc420d27000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).fill(0xc4201f7e00) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).ReadByte(0xc4201f7e00, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4209d3cb0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206f4280) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4201f7e60, 0x16e4b60, 0xc4206f42e0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4201f7e60, 0x16e4b60, 0xc4206f42e0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420eb6740, 0x16e4b60, 0xc4206f42e0, 0xc4206f42e0, 0xc42053e5c8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420eb6740, 0xc4206f42e0, 0x2d, 0xc423176bc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequestHeader(0xc42053e5a0, 0x1d72340, 0xc420eb6740, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a9d10, 0x8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequest(0xc42053e5a0, 0x1d72340, 0xc420eb6740, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).ServeRequest(0xc42053e5a0, 0x1d72340, 0xc420eb6740, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d0e0, 0xc420310180, 0xc42053e5a0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 207 [IO wait]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.runtime_pollWait(0x7f006138d7b0, 0x72, 0xc421192ce8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).wait(0xc420a10098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).waitRead(0xc420a10098, 0xc42099f000, 0x1000, 0x1000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*FD).Read(0xc420a10080, 0xc42099f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*netFD).Read(0xc420a10080, 0xc42099f000, 0x1000, 0x1000, 0xc42111bdd8, 0x1, 0xc42111be08) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*conn).Read(0xc42021a020, 0xc42099f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).Read(0xc420de06c0, 0xc420695ac0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: io.ReadAtLeast(0x1d59aa0, 0xc420de06c0, 0xc420695ac0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: io.ReadFull(0x1d59aa0, 0xc420de06c0, 0xc420695ac0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42023e420, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42023e420) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 119 [IO wait]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.runtime_pollWait(0x7f006138dd60, 0x72, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).wait(0xc4205c0b98, 0x72, 0xc42001c100, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).waitRead(0xc4205c0b98, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*FD).Accept(0xc4205c0b80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*netFD).accept(0xc4205c0b80, 0x0, 0xc4204c07a8, 0x721378) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*TCPListener).accept(0xc420204260, 0x0, 0xc4201fa9c1, 0x5) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*TCPListener).AcceptTCP(0xc420204260, 0x1, 0x0, 0x435248) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4207ce3c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 120 [IO wait]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.runtime_pollWait(0x7f006138dc90, 0x72, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).wait(0xc42055e018, 0x72, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).waitRead(0xc42055e018, 0xc4229c4000, 0x10000, 0x10000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*FD).ReadFrom(0xc42055e000, 0xc4229c4000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*netFD).readFrom(0xc42055e000, 0xc4229c4000, 0x10000, 0x10000, 0xeb, 0xeb, 0x1d62b40, 0x412539, 0xc421120e40) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*UDPConn).readFrom(0xc420204420, 0xc4229c4000, 0x10000, 0x10000, 0x101, 0xc421120e18, 0xc421120e90, 0x443cf7) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*UDPConn).ReadFrom(0xc420204420, 0xc4229c4000, 0x10000, 0x10000, 0x10000, 0x10000, 0x65b6, 0x986f7e0d, 0x2b96de0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4207ce3c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 121 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4207ce3c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 122 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4207ce3c0, 0x12a05f200, 0xc420eccc00, 0xc420f3f5c0, 0xc420f42b50) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:110 +0xef Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 123 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4207ce3c0, 0xc420f3f5c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 124 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4207ce3c0, 0x1dcd6500, 0xc420eccc60, 0xc420f3f5c0, 0xc420f42b60) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 125 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4202fd200) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 126 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4202fd200) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 127 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202fd200, 0x1bdd09a, 0x6, 0xc420bfe030) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 128 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202fd200, 0x1bdba6c, 0x5, 0xc420bfe060) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 129 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4202fd200, 0x1bdbc6a, 0x5, 0xc420bfe090) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 180 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420078480, 0xc42081b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).fill(0xc420d06ea0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).ReadByte(0xc420d06ea0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4209a16b0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4207573e0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d06f00, 0x16e4b60, 0xc420757460) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d06f00, 0x16e4b60, 0xc420757460, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e2df00, 0x16e4b60, 0xc420757460, 0xc420757460, 0xc42053e5c8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e2df00, 0xc420757460, 0x2d, 0xc423176dc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequestHeader(0xc42053e5a0, 0x1d72340, 0xc420e2df00, 0x30, 0x1, 0x3, 0x100000000000003, 0xc4200a9d10, 0x8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequest(0xc42053e5a0, 0x1d72340, 0xc420e2df00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).ServeRequest(0xc42053e5a0, 0x1d72340, 0xc420e2df00, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d0e0, 0xc420078480, 0xc42053e5a0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 131 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420ecccc0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420ecccc0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420ecccc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 132 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420eccd20, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420eccd20, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420eccd20) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 133 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420eccd80, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420eccd80, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420eccd80) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 134 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420eccde0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420eccde0, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420eccde0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 135 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420b65ce0, 0xc420010280) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:857 +0x116 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 136 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc420010280) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 137 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc420010280) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 138 [IO wait]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.runtime_pollWait(0x7f006138de30, 0x72, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).wait(0xc4205c0718, 0x72, 0xc420e4e000, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).waitRead(0xc4205c0718, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*FD).Accept(0xc4205c0700, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*netFD).accept(0xc4205c0700, 0x1, 0x1, 0xc420fe1f20) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*TCPListener).accept(0xc42000ee18, 0x406b4a, 0xc420f4a660, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*TCPListener).Accept(0xc42000ee18, 0xc420f4a660, 0x0, 0x3f800000, 0xc420a48240) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc420010280, 0x1d71840, 0xc420a48240) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 139 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc4203fd2c0, 0x3b9aca00, 0xc4204f4cc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 140 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc42053e000, 0x3b9aca00, 0xc4204f4cc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 141 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc4204ea7e0, 0x3b9aca00, 0xc4204f4cc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 142 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc420021790, 0x3b9aca00, 0xc4204f4cc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 143 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc420010280) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 144 [IO wait]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.runtime_pollWait(0x7f006138dbc0, 0x72, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).wait(0xc420e7a618, 0x72, 0xc4201feb00, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).waitRead(0xc420e7a618, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*FD).Accept(0xc420e7a600, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*netFD).accept(0xc420e7a600, 0x412d88, 0x30, 0x1a1fac0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*TCPListener).accept(0xc420205310, 0x5fea4a, 0x1a1fac0, 0xc4209eeb70) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*TCPListener).Accept(0xc420205310, 0xc42003e028, 0x1830320, 0x2ab0680, 0x1b7b4c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/http.(*Server).Serve(0xc4207ba1a0, 0x1d6fc00, 0xc420205310, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/http.Serve(0x1d6fc00, 0xc420205310, 0x1d5df00, 0xc4209eea80, 0x0, 0x7f004309c000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420a49400, 0x1d6fc00, 0xc420205310, 0xc420f43700, 0xc420b65e90) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 145 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420e7a680, 0xc420f43710, 0xc420f3f860) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 171 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b4000, 0xc420cf60c0, 0xc420df6de0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc42070bda0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 94 [IO wait]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.runtime_pollWait(0x7f006138da20, 0x72, 0xc42054ace8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).wait(0xc420d7ac98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).waitRead(0xc420d7ac98, 0xc420a86000, 0x1000, 0x1000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*FD).Read(0xc420d7ac80, 0xc420a86000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*netFD).Read(0xc420d7ac80, 0xc420a86000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*conn).Read(0xc42059a000, 0xc420a86000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).Read(0xc420086e40, 0xc420b66480, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: io.ReadAtLeast(0x1d59aa0, 0xc420086e40, 0xc420b66480, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: io.ReadFull(0x1d59aa0, 0xc420086e40, 0xc420b66480, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42023e210, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42023e210) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 95 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42023e210) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 96 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42023e210) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 184 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422bf60f0, 0xc422084780) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 150 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc4223fc0a0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 181 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc42039e380) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 173 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4220fcdc0, 0xc421190d10, 0x30, 0xf9866a) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4220fcdc0, 0x1c61988, 0xc420010280, 0x1d71840, 0xc420a48240) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d320, 0xc42000f4e8, 0xc421200eb0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d320, 0xc42000f4e8, 0xc421200eb0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 174 [IO wait]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.runtime_pollWait(0x7f006138d540, 0x72, 0xc421122ce8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).wait(0xc420c1f898, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).waitRead(0xc420c1f898, 0xc420b2a000, 0x1000, 0x1000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*FD).Read(0xc420c1f880, 0xc420b2a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*netFD).Read(0xc420c1f880, 0xc420b2a000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*conn).Read(0xc42000f4e8, 0xc420b2a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).Read(0xc420558060, 0xc4214186c0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: io.ReadAtLeast(0x1d59aa0, 0xc420558060, 0xc4214186c0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: io.ReadFull(0x1d59aa0, 0xc420558060, 0xc4214186c0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4220fcdc0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4220fcdc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 202 [runnable]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc4203fd2c0, 0xc42071de40, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:225 +0x33e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc4203fd2c0, 0xc42071de40) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc420010280, 0xc4205c6e01, 0xc4205c6ec0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc420010280, 0xc420dc7ce0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc420010280, 0xc420dc7ce0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc420b66200, 0xc420010280, 0xc420dc7ce0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 203 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4200b4000, 0xc420cf6000, 0xc420dc7da0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc4206f4aa0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 204 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc420eb7dc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 205 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b4000, 0xc420cf6000, 0x1d6eb80, 0xc420eb7dc0, 0xc420d902a0, 0xc420d90360) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc420f4c0c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 182 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4200b4000, 0xc420cf60c0, 0x1d6eb80, 0xc42039e380, 0xc4209eb2c0, 0xc4209eb3e0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4200b4000, 0xc42039e3c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 208 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42023e420) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 209 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42023e420) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 210 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420cf6180, 0xc420873000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).fill(0xc420cccb40) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).ReadByte(0xc420cccb40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a4b3e0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4212822e0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420cccba0, 0x16e4b60, 0xc421282340) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420cccba0, 0x16e4b60, 0xc421282340, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420a6fec0, 0x16e4b60, 0xc421282340, 0xc421282340, 0xc42138e2f8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420a6fec0, 0xc421282340, 0x2d, 0xc423176b00) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequestHeader(0xc42138e2d0, 0x1d72340, 0xc420a6fec0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200a9d10, 0x8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequest(0xc42138e2d0, 0x1d72340, 0xc420a6fec0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).ServeRequest(0xc42138e2d0, 0x1d72340, 0xc420a6fec0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d0e0, 0xc420cf6180, 0xc42138e2d0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 211 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4200780c0, 0xc42078a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).fill(0xc420ccce40) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).ReadByte(0xc420ccce40, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a4b560, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4212839e0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420cccea0, 0x16e4b60, 0xc421283a40) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420cccea0, 0x16e4b60, 0xc421283a40, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420a6ffc0, 0x16e4b60, 0xc421283a40, 0xc421283a40, 0xc42053e5c8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420a6ffc0, 0xc421283a40, 0x2d, 0xc422326680) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequestHeader(0xc42053e5a0, 0x1d72340, 0xc420a6ffc0, 0x0, 0x0, 0x3, 0x100000000000003, 0xc4200a9d10, 0x8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequest(0xc42053e5a0, 0x1d72340, 0xc420a6ffc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).ServeRequest(0xc42053e5a0, 0x1d72340, 0xc420a6ffc0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d0e0, 0xc4200780c0, 0xc42053e5a0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 183 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422bf60f0, 0xc4220846c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 213 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4213c20b0, 0xc420062d10, 0x30, 0xf9866a) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4213c20b0, 0x1c61988, 0xc420010280, 0x1d71840, 0xc420a48240) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d320, 0xc42000e028, 0xc421adb860) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d320, 0xc42000e028, 0xc421adb860) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 214 [IO wait]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.runtime_pollWait(0x7f006138d610, 0x72, 0xc42118fce8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).wait(0xc420d7a198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*pollDesc).waitRead(0xc420d7a198, 0xc420d26000, 0x1000, 0x1000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: internal/poll.(*FD).Read(0xc420d7a180, 0xc420d26000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*netFD).Read(0xc420d7a180, 0xc420d26000, 0x1000, 0x1000, 0xc42118fdd8, 0x1, 0xc42118fe08) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net.(*conn).Read(0xc42000e028, 0xc420d26000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).Read(0xc421997b00, 0xc420470a70, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: io.ReadAtLeast(0x1d59aa0, 0xc421997b00, 0xc420470a70, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: io.ReadFull(0x1d59aa0, 0xc421997b00, 0xc420470a70, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4213c20b0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4213c20b0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 215 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4213c20b0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 216 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4213c20b0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 217 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420238000, 0xc420eb3000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).fill(0xc420d06600) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: bufio.(*Reader).ReadByte(0xc420d06600, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420b01a40, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4206d0740) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420d06660, 0x16e4b60, 0xc4206d07a0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420d06660, 0x16e4b60, 0xc4206d07a0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4214c5f40, 0x16e4b60, 0xc4206d07a0, 0xc4206d07a0, 0xc421adb928) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4214c5f40, 0xc4206d07a0, 0x2d, 0xc421877040) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequestHeader(0xc421adb900, 0x1d72340, 0xc4214c5f40, 0x60, 0x2, 0x3, 0x100000000000003, 0xc4200a9d10, 0x8) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).readRequest(0xc421adb900, 0x1d72340, 0xc4214c5f40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).ServeRequest(0xc421adb900, 0x1d72340, 0xc4214c5f40, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d0e0, 0xc420238000, 0xc421adb900) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 175 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4220fcdc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 176 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4220fcdc0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 177 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420de1980, 0xc421c75700, 0x20, 0x20, 0x169d4e0, 0xc42072f7e0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4213f7cb0, 0x1d718c0, 0xc420de1980, 0x0, 0xc420de1900) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420010280, 0xc421c75950, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42106a060, 0xc42072a500, 0xc4213f7a10, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: reflect.Value.call(0xc4205c41e0, 0xc42000f7f8, 0x13, 0x1bda9a0, 0x4, 0xc421c75d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: reflect.Value.Call(0xc4205c41e0, 0xc42000f7f8, 0x13, 0xc421c75d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*service).call(0xc421147080, 0xc421200f50, 0xc420aab948, 0x0, 0xc420d17d00, 0xc420943980, 0x19758e0, 0xc42072a500, 0x16, 0x169e020, ...) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: net/rpc.(*Server).ServeRequest(0xc421200f50, 0x1d72340, 0xc421147140, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420010280, 0x1d71840, 0xc420a48240, 0x1d7d0e0, 0xc420cf6540, 0xc421200f50) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 226 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420ce8780) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 227 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420ce8780) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 228 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc42053e000, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc420010280) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 230 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc4204ea7e0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 231 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc4204ea7e0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 232 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc4221facf0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 233 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42242be80, 0xc423192ce8, 0x20, 0x20, 0x18, 0xc422c6a5e0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422b5fec0, 0x1d71840, 0xc42242be80, 0xc422c6a5e0, 0x184987) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4205c6ec0, 0xc421c6fed0, 0x184987, 0x1d71840, 0xc42242be80, 0xc42055ee60, 0xc4221facc0, 0xc420a32b40, 0x24, 0x2bba400) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc42055ee00, 0x1d71840, 0xc42242be80, 0x184987, 0xc422f34e80, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc42055ee00, 0x1d71840, 0xc42242be80) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 234 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4223fe040, 0xc4214e3770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc423261290, 0x1d71840, 0xc4223fe040, 0x0, 0x1849b5) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4205c6ec0, 0xc4214e3960, 0x1849b5, 0x1d71840, 0xc4223fe040, 0x552bde, 0xc4214e3950, 0xc4214e3970, 0xf33ba5, 0xc4204a1718) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc4204a16c0, 0x1d71840, 0xc4223fe040, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc4204a16c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 235 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc4223fe000, 0xc42307fa78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4232614a0, 0x1d71840, 0xc4223fe000, 0xc423261590, 0x1833a4) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4205c6ec0, 0xc42307fc68, 0x1833a4, 0x1d71840, 0xc4223fe000, 0x552bde, 0xc4200a9db0, 0x2, 0xc422ba8380, 0x40) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc4223fe5c0, 0x1833a4, 0x3b, 0xc421125de8, 0x1, 0x1) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc4223fe5c0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 236 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc4223fe640) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 237 [select]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4207ce5a0, 0x1d71840, 0xc4223fe000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: goroutine 238 [runnable]: Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc420feb220, 0x7f005c03a000, 0xc422f35000, 0x1, 0x0, 0x0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:248 +0x2c1 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc420feb220, 0x7f005c03a000, 0xc422f35000, 0x7f005c03a000, 0xc422f35000) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsImpl(0xc4223fc0a0, 0xc4221fb890, 0xc421f91ee0, 0x351d01858d575e4b, 0xc4201b4e28, 0x351d01858d575e4b, 0x1afe340, 0xc4221fb8a0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:756 +0x89 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).(github.com/hashicorp/nomad/nomad/deploymentwatcher.getAllocsImpl)-fm(0xc4221fb890, 0xc421f91ee0, 0xc4203ce300, 0xc422400278, 0xc420010200, 0x1be6698, 0xc) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x3e Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc4205c6ec0, 0xc42318ef20, 0x1849b5, 0x1d71840, 0xc422f35000, 0x0, 0x0, 0x0, 0xc4200877a0, 0x4) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:149 +0x114 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc4223fc0a0, 0x1849b5, 0x0, 0x0, 0xc4201b4fc8, 0xfb15f8, 0xc420eccde0, 0x1dcd6500) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc4223fc0a0, 0x1849b5, 0xc421ef1aa0) Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:29:45 ip-172-24-20-236 nomad.sh[26133]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:29:45 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:29:45 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:29:45 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: ==> Starting Nomad agent... Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: ==> Nomad agent configuration: Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: Client: false Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: Log Level: INFO Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: Region: amrv01 (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: Server: true Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: Version: 0.8.4 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.220970 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.221052 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: Attempting re-join to previously known node: consulnomad-staging-01.asv01: 172.27.0.136:4648 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.358775 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.358797 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.358811 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.358891 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.358904 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.575513 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.575537 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.575564 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.575576 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.575587 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.575598 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.575820 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.575858 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.575872 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [WARN] memberlist: Refuting an alive message Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.576108 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.576124 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.576135 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.576147 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46 [INFO] serf: Re-joined to previously known node: consulnomad-staging-01.asv01: 172.27.0.136:4648 Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.576328 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:29:46 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:46.576340 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:47 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:47 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:48 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:48 [WARN] raft: Failed to get previous log: 1598484 log not found (last: 1598482) Aug 23 15:29:49 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:49 [INFO] serf: EventMemberUpdate: ip-172-24-10-89.amrv01 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:50 [WARN] raft: Heartbeat timeout from "172.24.0.121:4647" reached, starting election Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:50 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 80 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:50 [INFO] raft: Election won. Tally: 2 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:50 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:50 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:50 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:50.170550 [INFO] nomad: cluster leadership acquired Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:50 [WARN] raft: AppendEntries to {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} rejected, sending older logs (next: 1598485) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: 2018/08/23 15:29:50 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: fatal error: concurrent map read and map write Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 272 [running]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: runtime.throw(0x1c0f7a3, 0x21) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc422de1d20 sp=0xc422de1d00 pc=0x42d161 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: runtime.mapaccess2(0x1844be0, 0xc4204231d0, 0xc422de1de0, 0xc4204570d8, 0x8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/hashmap.go:409 +0x225 fp=0xc422de1d68 sp=0xc422de1d20 pc=0x409c75 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/lib.(*DelayHeap).Remove(0xc420424200, 0x1d6dd40, 0xc4204570d8, 0x0, 0xed3109fbe) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:156 +0xdc fp=0xc422de1e50 sp=0xc422de1d68 pc=0xf215ac Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc42035c1e0, 0x1d71840, 0xc422c04500) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:766 +0x277 fp=0xc422de1fc8 sp=0xc422de1e50 pc=0xf56bb7 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: runtime.goexit() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc422de1fd0 sp=0xc422de1fc8 pc=0x45c711 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 1 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc420357880, 0xc420499a40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc420357880, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc42031b7c0, 0xc42031b7c0, 0xc4204c8b20, 0x37) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204eff68) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420094058) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: main.main() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 5 [syscall]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: os/signal.signal_recv(0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: os/signal.loop() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by os/signal.init.0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 229 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420238210) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 92 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc4203b8000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc4203b8000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc4203b8000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203b8000, 0xc4203e1f60) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 276 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421cdafd0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 26 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc420023a80) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 13 [select, locked to thread]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: runtime.selectgo(0xc420055f50, 0xc4202ec960) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: runtime.ensureSigM.func1() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: runtime.goexit() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 27 [sleep]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: time.Sleep(0x3b9aca00) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc420534000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 28 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420536300) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 54 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc42018c840, 0x1d71840, 0xc42018c880) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 14 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4202fe180) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 227 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c470, 0x72, 0xc42127cce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420b62518, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420b62518, 0xc420dcc000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc420b62500, 0xc420dcc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc420b62500, 0xc420dcc000, 0x1000, 0x1000, 0xc42127cdd8, 0x0, 0xc42127ce08) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc420214300, 0xc420dcc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc420087d40, 0xc420f4e380, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc420087d40, 0xc420f4e380, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc420087d40, 0xc420f4e380, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420238210, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420238210) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 16 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc420381a00) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc420423ad0, 0xc4203220b0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 40 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121cf00, 0x72, 0xc4200629a8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420142318, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420142318, 0xc420476000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc420142300, 0xc420476000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc420142300, 0xc420476000, 0x1000, 0x1000, 0x458600, 0xc420f7e300, 0x4) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc4204560c0, 0xc420476000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.(*persistConn).Read(0xc42033e240, 0xc420476000, 0x1000, 0x1000, 0xc420062b98, 0x405675, 0xc4200945a0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).fill(0xc4200866c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Peek(0xc4200866c0, 0x1, 0x0, 0x0, 0x1, 0xc420d0fb60, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.(*persistConn).readLoop(0xc42033e240) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by net/http.(*Transport).dialConn Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 41 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.(*persistConn).writeLoop(0xc42033e240) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by net/http.(*Transport).dialConn Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 94 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc4203b8000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203b8000, 0xc4203e1f80) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 203 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420238420) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 186 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121caf0, 0x72, 0xc42050cce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420e8f798, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420e8f798, 0xc42126a000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc420e8f780, 0xc42126a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc420e8f780, 0xc42126a000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc42000e968, 0xc42126a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc420ea7500, 0xc4202df4a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc420ea7500, 0xc4202df4a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc420ea7500, 0xc4202df4a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420534580, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420534580) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 226 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420238210, 0xc42136dd10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420238210, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420214300, 0xc420203540) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420214300, 0xc420203540) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 96 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc420a241a0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 97 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121ce30, 0x72, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420bf4098, 0x72, 0xc420082100, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420bf4098, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Accept(0xc420bf4080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).accept(0xc420bf4080, 0xc42000e940, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*TCPListener).accept(0xc4202142c0, 0x458e40, 0xc4203c4f90, 0xc4203c4f98) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*TCPListener).AcceptTCP(0xc4202142c0, 0x1c63658, 0xc420cf83c0, 0xc42000e940) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:29:50 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc420cf83c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 95 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc420c11620) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 82 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc4202023c0, 0x1c636a0, 0xc420404840, 0x1d71840, 0xc42018d580) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc420404840) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 180 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4205340b0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 126 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121ca20, 0x72, 0xc4200619a8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420a28818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420a28818, 0xc420cdc000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc420a28800, 0xc420cdc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc420a28800, 0xc420cdc000, 0x1000, 0x1000, 0x458600, 0xc4202f0600, 0x4) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc42000e4e0, 0xc420cdc000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.(*persistConn).Read(0xc420d46000, 0xc420cdc000, 0x1000, 0x1000, 0xc420061b98, 0x405675, 0xc420df90e0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).fill(0xc4211ff020) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Peek(0xc4211ff020, 0x1, 0x0, 0x0, 0x1, 0xc420e65e60, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.(*persistConn).readLoop(0xc420d46000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by net/http.(*Transport).dialConn Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 228 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420238210) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 93 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc4203b8000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:29:50 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203b8000, 0xc4203e1f70) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 130 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121cc90, 0x72, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420bf4118, 0x72, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420bf4118, 0xc42332a000, 0x10000, 0x10000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).ReadFrom(0xc420bf4100, 0xc42332a000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).readFrom(0xc420bf4100, 0xc42332a000, 0x10000, 0x10000, 0xe9, 0xe9, 0x1d62b40, 0x412539, 0xc421371e40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*UDPConn).readFrom(0xc4202142e0, 0xc42332a000, 0x10000, 0x10000, 0x101, 0xc421371e18, 0xc421371e90, 0x443cf7) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*UDPConn).ReadFrom(0xc4202142e0, 0xc42332a000, 0x10000, 0x10000, 0x10000, 0x10000, 0x5d93, 0xf7ed17e2, 0x2b96de0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc420cf83c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 131 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc420cf83c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 132 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420cf83c0, 0x12a05f200, 0xc420e5d740, 0xc420deda40, 0xc420322ef0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:110 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 133 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc420cf83c0, 0xc420deda40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 134 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc420cf83c0, 0x1dcd6500, 0xc420e5d7a0, 0xc420deda40, 0xc420322f40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 135 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4204aac00) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 136 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4204aac00) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 137 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204aac00, 0x1bdd09a, 0x6, 0xc420c11770) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 138 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204aac00, 0x1bdba6c, 0x5, 0xc420c117a0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 139 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204aac00, 0x1bdbc6a, 0x5, 0xc420c117d0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 185 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420534580, 0xc420fd9d10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420534580, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc42000e968, 0xc421201d10) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc42000e968, 0xc421201d10) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 141 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc42049d180, 0x1be6698, 0xc, 0x1d76b80, 0xc421dedbc0, 0x19746e0, 0xc421dedbc0, 0x181f280, 0xc422c54f80, 0xc4202fecc0, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:393 +0x3bc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc420214038, 0xc421dedbc0, 0xc422c54f80, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.call(0xc4202fecc0, 0xc4202144e8, 0x13, 0x1bda9a0, 0x4, 0xc421c9dd18, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.Call(0xc4202fecc0, 0xc4202144e8, 0x13, 0xc421c9dd18, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*service).call(0xc420402280, 0xc420202190, 0xc4227f9ee0, 0x0, 0xc42021ce00, 0xc420632060, 0x19746e0, 0xc421dedbc0, 0x16, 0x181f280, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc420202190, 0x1d71b80, 0xc422c54f40, 0x5b7ea8be, 0xc40656f237) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc42049d180, 0x1be6698, 0xc, 0x19746e0, 0xc4211fe840, 0x181f280, 0xc420b16100, 0x1d59ee0, 0xc4206e2140) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e5d800, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:158 +0x1bb Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e5d800) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 142 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e5d860, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e5d860, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e5d860) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 143 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc423222300, 0xc421a81000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).fill(0xc422d1ccc0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).ReadByte(0xc422d1ccc0, 0xc420040000, 0xc421e27998, 0xc421e279a0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42283e1b0, 0xc422c21410) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422fd92e0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422d1cd20, 0x16e4ba0, 0xc42283e3c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422d1cd20, 0x16e4ba0, 0xc42283e3c0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421e279c0, 0x16e4ba0, 0xc42283e3c0, 0x1, 0xc42283e3c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadResponseHeader(0xc421e279c0, 0xc42283e3c0, 0x19746e0, 0xc42242bb00) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:74 +0x41 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.CallWithCodec(0x1d72300, 0xc421e279c0, 0x1be6698, 0xc, 0x19746e0, 0xc42242bb00, 0x181f280, 0xc4225a7d40, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/client.go:29 +0xff Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).RPC(0xc4202fe180, 0xc4204b6c81, 0x6, 0x1d62b00, 0xc421097d70, 0x1, 0x1be6698, 0xc, 0x19746e0, 0xc42242bb00, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:422 +0x19b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).forwardLeader(0xc42049d180, 0xc420d82ee0, 0x1be6698, 0xc, 0x19746e0, 0xc42242bb00, 0x181f280, 0xc4225a7d40, 0xc420d08e00, 0x1947b20) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:434 +0xd1 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).forward(0xc42049d180, 0x1be6698, 0xc, 0x1d76b80, 0xc42242bb00, 0x19746e0, 0xc42242bb00, 0x181f280, 0xc4225a7d40, 0xc4202fecc0, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:383 +0x4ce Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Eval).Dequeue(0xc420214038, 0xc42242bb00, 0xc4225a7d40, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_endpoint.go:75 +0xa6 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.call(0xc4202fecc0, 0xc4202144e8, 0x13, 0x1bda9a0, 0x4, 0xc42127bd18, 0x3, 0x3, 0x19e52c0, 0x19, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.Call(0xc4202fecc0, 0xc4202144e8, 0x13, 0xc42127bd18, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*service).call(0xc420402280, 0xc420202190, 0xc4217ac090, 0x0, 0xc42021ce00, 0xc420604040, 0x19746e0, 0xc42242bb00, 0x16, 0x181f280, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc420202190, 0x1d71b80, 0xc4225a7d00, 0x5b7ea8bd, 0xc40f9866e3) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).RPC(0xc42049d180, 0x1be6698, 0xc, 0x19746e0, 0xc420b1a000, 0x181f280, 0xc420d04000, 0x1d59ee0, 0xc420207e40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1394 +0xc4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e5d8c0, 0x1dcd6500, 0x2ed0cdaed, 0x1000, 0x309f, 0x0, 0x30a1) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:158 +0x1bb Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e5d8c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 144 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc420e5d920, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc420e5d920, 0x1dcd6500, 0x2ed0cdaed, 0x1000, 0x309f, 0x0, 0x30a1) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc420e5d920) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 145 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c6e0, 0x77, 0xc420f7e300) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420e7e418, 0x77, 0xc420fe5000, 0xc420fe5028, 0x436544) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitWrite(0xc420e7e418, 0xc420fe5000, 0x28, 0x611b26) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).WaitWrite(0xc420e7e400, 0x1c65988, 0x1d718c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:440 +0x37 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).connect(0xc420e7e400, 0x1d718c0, 0xc420185b60, 0x0, 0x0, 0x1d5c800, 0xc422574740, 0x0, 0x0, 0x0, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:152 +0x299 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).dial(0xc420e7e400, 0x1d718c0, 0xc420185b60, 0x1d7a200, 0x0, 0x1d7a200, 0xc421bc90e0, 0xc420fe5248, 0x62f2be) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/sock_posix.go:142 +0xe9 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.socket(0x1d718c0, 0xc420185b60, 0x1bda392, 0x3, 0x2, 0x1, 0x0, 0x0, 0x1d7a200, 0x0, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/sock_posix.go:93 +0x1a6 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.internetSocket(0x1d718c0, 0xc420185b60, 0x1bda392, 0x3, 0x1d7a200, 0x0, 0x1d7a200, 0xc421bc90e0, 0x1, 0x0, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/ipsock_posix.go:141 +0x129 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.doDialTCP(0x1d718c0, 0xc420185b60, 0x1bda392, 0x3, 0x0, 0xc421bc90e0, 0x2bb8aa0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/tcpsock_posix.go:62 +0xb9 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.dialTCP(0x1d718c0, 0xc420185b60, 0x1bda392, 0x3, 0x0, 0xc421bc90e0, 0x7f356606c, 0x2b96de0, 0x6fc238262) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/tcpsock_posix.go:58 +0xe4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.dialSingle(0x1d718c0, 0xc420185b60, 0xc420e7e200, 0x1d62b00, 0xc421bc90e0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/dial.go:547 +0x375 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.dialSerial(0x1d718c0, 0xc420185b60, 0xc420e7e200, 0xc422baa700, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/dial.go:515 +0x22d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*Dialer).DialContext(0xc420fe5848, 0x1d71880, 0xc42003e028, 0x1bda392, 0x3, 0xc4225746e0, 0x11, 0x0, 0x0, 0x0, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/dial.go:397 +0x678 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*Dialer).Dial(0xc420fe5848, 0x1bda392, 0x3, 0xc4225746e0, 0x11, 0x20, 0x20, 0x17503e0, 0x2ade3c8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/dial.go:320 +0x75 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc420cf83c0, 0xc42245bd90, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:758 +0x14e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc420cf83c0, 0xc42245bd90, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc420cf83c0, 0xc420446200, 0x14, 0x20, 0x3, 0x3, 0xc4208458d0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc4204aac00, 0xc420446200, 0x14, 0x20, 0xc420845801, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc42049d180, 0xc420446200, 0x14, 0x20, 0xc4208458d0, 0x10, 0xc420516da0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc420fd7740, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc4210977d0, 0xc42049d180) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 146 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc42049d180) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 147 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc42049d180) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 148 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121cd60, 0x72, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420217818, 0x72, 0xc420b16000, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420217818, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Accept(0xc420217800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).accept(0xc420217800, 0x1, 0x1, 0xc420508f20) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*TCPListener).accept(0xc420214e98, 0x406b4a, 0xc420e2e5a0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*TCPListener).Accept(0xc420214e98, 0xc420e2e5a0, 0x0, 0x3f800000, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 149 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc42035c1e0, 0x3b9aca00, 0xc420262420) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 150 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc420202140, 0x3b9aca00, 0xc420262420) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 151 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc4204062d0, 0x3b9aca00, 0xc420262420) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 152 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc420381a00, 0x3b9aca00, 0xc420262420) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 153 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc42049d180) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 154 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121cbc0, 0x72, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420d9e498, 0x72, 0xc42018c000, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420d9e498, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Accept(0xc420d9e480, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).accept(0xc420d9e480, 0x412d88, 0x30, 0x1a1fac0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*TCPListener).accept(0xc420214db0, 0x5fea4a, 0x1a1fac0, 0xc420b722a0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*TCPListener).Accept(0xc420214db0, 0xc42003e028, 0x1830320, 0x2ab0680, 0x1b7b4c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.(*Server).Serve(0xc4200aec30, 0x1d6fc00, 0xc420214db0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.Serve(0x1d6fc00, 0xc420214db0, 0x1d5df00, 0xc420b721e0, 0x1c632f0, 0xc420ff2390) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc420cfcb00, 0x1d6fc00, 0xc420214db0, 0xc4203239d0, 0xc421097890) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 155 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc420d9e500, 0xc420323a10, 0xc420e64240) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 156 [sleep]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: time.Sleep(0x5dfc10753) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/command/agent.(*Command).setupAgent.func1(0xc420357880, 0xc420d9e500) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:415 +0x3e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/command/agent.(*Command).setupAgent Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:414 +0x32c Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 46 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4205340b0, 0xc420fd8d10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4205340b0, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420456190, 0xc420e02230) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420456190, 0xc420e02230) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 179 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4205340b0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 127 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.(*persistConn).writeLoop(0xc420d46000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by net/http.(*Transport).dialConn Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 164 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c950, 0x72, 0xc420fea9a8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420b62098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420b62098, 0xc420d45000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc420b62080, 0xc420d45000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc420b62080, 0xc420d45000, 0x1000, 0x1000, 0xc42003e028, 0x1bda392, 0x3) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc4202081c8, 0xc420d45000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.(*persistConn).Read(0xc420d48000, 0xc420d45000, 0x1000, 0x1000, 0x405486, 0xc420feab98, 0x42bc79) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).fill(0xc420ceb8c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Peek(0xc420ceb8c0, 0x1, 0xc420dd81e0, 0xc42004ec80, 0xc42004ebf8, 0x412d88, 0x60) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.(*persistConn).readLoop(0xc420d48000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by net/http.(*Transport).dialConn Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 165 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/http.(*persistConn).writeLoop(0xc420d48000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by net/http.(*Transport).dialConn Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 178 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c880, 0x72, 0xc420febce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc42090e198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc42090e198, 0xc42100e000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc42090e180, 0xc42100e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc42090e180, 0xc42100e000, 0x1000, 0x1000, 0xc420febdd8, 0x0, 0xc420febe08) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc420456190, 0xc42100e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc4211ff1a0, 0xc420620280, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc4211ff1a0, 0xc420620280, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc4211ff1a0, 0xc420620280, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4205340b0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4205340b0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 202 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420238420) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 187 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420534580) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 188 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420534580) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 189 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420c0eea0, 0xc420fe1700, 0x20, 0x20, 0x169d4e0, 0xc420619380) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4209abd10, 0x1d718c0, 0xc420c0eea0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42049d180, 0xc420fe1950, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc42126f2c0, 0xc420e7e000, 0xc421458570, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.call(0xc420396480, 0xc42000ec70, 0x13, 0x1bda9a0, 0x4, 0xc420fe1d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.Call(0xc420396480, 0xc42000ec70, 0x13, 0xc420fe1d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*service).call(0xc420b17e40, 0xc421201db0, 0xc420506a70, 0x0, 0xc421272080, 0xc420516960, 0x19758e0, 0xc420e7e000, 0x16, 0x169e020, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc421201db0, 0x1d72340, 0xc420b17e80, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d0e0, 0xc420c20240, 0xc421201db0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 190 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420534630, 0xc42127dd10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420534630, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc42000ecb8, 0xc421201ea0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc42000ecb8, 0xc421201ea0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 191 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c7b0, 0x72, 0xc421279ce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc421272598, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc421272598, 0xc421275000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc421272580, 0xc421275000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc421272580, 0xc421275000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc42000ecb8, 0xc421275000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc42126f9e0, 0xc4206202e0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc42126f9e0, 0xc4206202e0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc42126f9e0, 0xc4206202e0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420534630, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420534630) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 192 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420534630) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 193 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420534630) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 172 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420238160, 0xc420fded10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420238160, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc4204560c8, 0xc420202410) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc4204560c8, 0xc420202410) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 239 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422a56000, 0xc42127ad10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422a56000, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc42000f228, 0xc422805040) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc42000f228, 0xc422805040) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 218 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422d1c5a0, 0xc42102d700, 0x20, 0x20, 0x169d4e0, 0xc420619380) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422f9aed0, 0x1d718c0, 0xc422d1c5a0, 0x0, 0xc422d1c500) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42049d180, 0xc42102d950, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4212817a0, 0xc422a69400, 0xc422f9ade0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.call(0xc420396480, 0xc42000efc0, 0x13, 0x1bda9a0, 0x4, 0xc42102dd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.Call(0xc420396480, 0xc42000efc0, 0x13, 0xc42102dd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*service).call(0xc421282340, 0xc421201f40, 0xc4231ef3e8, 0x0, 0xc421286e80, 0xc4205de7a0, 0x19758e0, 0xc422a69400, 0x16, 0x169e020, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc421201f40, 0x1d72340, 0xc42206de00, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d0e0, 0xc420c20180, 0xc421201f40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 274 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c540, 0x72, 0xc422bd9ce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420bf4598, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420bf4598, 0xc422365000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc420bf4580, 0xc422365000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc420bf4580, 0xc422365000, 0x1000, 0x1000, 0xc422bd9dd8, 0x0, 0xc422bd9e08) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc420215738, 0xc422365000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc4224e6780, 0xc4220ff230, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc4224e6780, 0xc4220ff230, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc4224e6780, 0xc4220ff230, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421cdafd0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421cdafd0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 257 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421cdafd0, 0xc421029d10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421cdafd0, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420215738, 0xc420219b30) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420215738, 0xc420219b30) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 241 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422a56000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 240 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c130, 0x72, 0xc421277ce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc422894498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc422894498, 0xc421614000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc422894480, 0xc421614000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc422894480, 0xc421614000, 0x1000, 0x1000, 0xc4203c45d8, 0x1, 0xc4203c4608) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc42000f228, 0xc421614000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc422946f60, 0xc4229486f0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc422946f60, 0xc4229486f0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc422946f60, 0xc4229486f0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422a56000, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422a56000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 173 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c3a0, 0x72, 0xc421361ce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc4213a8018, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc4213a8018, 0xc42100f000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc4213a8000, 0xc42100f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc4213a8000, 0xc42100f000, 0x1000, 0x1000, 0xc421361dd8, 0x0, 0xc421361e08) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc4204560c8, 0xc42100f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc420e5c120, 0xc420020fd0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc420e5c120, 0xc420020fd0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc420e5c120, 0xc420020fd0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420238160, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420238160) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 174 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420238160) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 175 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420238160) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 201 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c2d0, 0x72, 0xc42135fce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420b62118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420b62118, 0xc420dcd000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc420b62100, 0xc420dcd000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc420b62100, 0xc420dcd000, 0x1000, 0x1000, 0xc42135fdd8, 0x0, 0xc42135fe08) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc420215008, 0xc420dcd000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc4210d60c0, 0xc420d00690, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc4210d60c0, 0xc420d00690, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc4210d60c0, 0xc420d00690, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420238420, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420238420) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 200 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420238420, 0xc420066d10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420238420, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420215008, 0xc420608050) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420215008, 0xc420608050) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 244 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c200, 0x72, 0xc421360ce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420a28118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420a28118, 0xc420d44000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc420a28100, 0xc420d44000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc420a28100, 0xc420d44000, 0x1000, 0x1000, 0xc421360dd8, 0x0, 0xc421360e08) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc42000f218, 0xc420d44000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc4213994a0, 0xc4212a4b10, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc4213994a0, 0xc4212a4b10, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc4213994a0, 0xc4212a4b10, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421cda000, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421cda000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 245 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421cda000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 246 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421cda000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 258 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422a56000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 259 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4205363c0, 0xc422a6e000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).fill(0xc422a673e0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).ReadByte(0xc422a673e0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422a62f90, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42252fdc0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc422a67440, 0x16e4b60, 0xc42252fe20) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc422a67440, 0x16e4b60, 0xc42252fe20, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc42292f0c0, 0x16e4b60, 0xc42252fe20, 0xc42252fe20, 0xc422805108) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc42292f0c0, 0xc42252fe20, 0x2d, 0xc420de0ba0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).readRequestHeader(0xc4228050e0, 0x1d72340, 0xc42292f0c0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42002b590, 0x8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).readRequest(0xc4228050e0, 0x1d72340, 0xc42292f0c0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc4228050e0, 0x1d72340, 0xc42292f0c0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d0e0, 0xc4205363c0, 0xc4228050e0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 275 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421cdafd0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 261 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421cda0b0, 0xc42135ed10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421cda0b0, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc4202153a0, 0xc420218870) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc4202153a0, 0xc420218870) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 262 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c610, 0x72, 0xc420fe9ce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc42210c298, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc42210c298, 0xc422d48000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc42210c280, 0xc422d48000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc42210c280, 0xc422d48000, 0x1000, 0x1000, 0xfa, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc4202153a0, 0xc422d48000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc4211fec00, 0xc4231ef550, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc4211fec00, 0xc4231ef550, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc4211fec00, 0xc4231ef550, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421cda0b0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421cda0b0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 263 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421cda0b0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 264 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421cda0b0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 270 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc4209ae640) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 271 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc420202140, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc42049d180) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 267 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4224e6120, 0xc421c9b700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422a24300, 0x1d718c0, 0xc4224e6120, 0x0, 0xc4224e6100) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42049d180, 0xc421c9b950, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420184600, 0xc420bf4380, 0xc422a241b0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.call(0xc420396480, 0xc420456850, 0x13, 0x1bda9a0, 0x4, 0xc420feed48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.Call(0xc420396480, 0xc420456850, 0x13, 0xc420feed48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*service).call(0xc42104a400, 0xc420203310, 0xc420a7b368, 0x0, 0xc420ed7180, 0xc420f72820, 0x19758e0, 0xc420bf4380, 0x16, 0x169e020, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc420203310, 0x1d72340, 0xc4225a6680, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d0e0, 0xc4232220c0, 0xc420203310) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 269 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc4209ae640) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 253 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc422720a20, 0xc421c97700, 0x20, 0x20, 0x169d4e0, 0xc420619380) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4229ed9e0, 0x1d718c0, 0xc422720a20, 0x0, 0xc422720a00) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42049d180, 0xc421c97950, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420c0faa0, 0xc420982b00, 0xc4229ed8c0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.call(0xc420396480, 0xc420214fb8, 0x13, 0x1bda9a0, 0x4, 0xc422bd8d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.Call(0xc420396480, 0xc420214fb8, 0x13, 0xc422bd8d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*service).call(0xc420cfd240, 0xc4202035e0, 0xc422a14c38, 0x0, 0xc42103e880, 0xc42289d680, 0x19758e0, 0xc420982b00, 0x16, 0x169e020, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc4202035e0, 0x1d72340, 0xc421e26340, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d0e0, 0xc420232b40, 0xc4202035e0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 222 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42242a000, 0xc422c1f700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422887e00, 0x1d718c0, 0xc42242a000, 0x0, 0xc42242a000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42049d180, 0xc422c1f950, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4210d7ec0, 0xc420983880, 0xc422887ce0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.call(0xc420396480, 0xc420215320, 0x13, 0x1bda9a0, 0x4, 0xc420feed48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.Call(0xc420396480, 0xc420215320, 0x13, 0xc420feed48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*service).call(0xc421282540, 0xc4206080f0, 0xc4224fa5e8, 0x0, 0xc4210ae380, 0xc4228622e0, 0x19758e0, 0xc420983880, 0x16, 0x169e020, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc4206080f0, 0x1d72340, 0xc421dc0100, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d0e0, 0xc4205366c0, 0xc4206080f0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 306 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422a57ce0, 0xc420fdfd10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422a57ce0, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc42000fff0, 0xc4224d9a90) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc42000fff0, 0xc4224d9a90) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 224 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e121c060, 0x72, 0xc422e1cce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc4210aea98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc4210aea98, 0xc421a80000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc4210aea80, 0xc421a80000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc4210aea80, 0xc421a80000, 0x1000, 0x1000, 0xc422e1cdd8, 0x0, 0xc422e1ce08) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc420208100, 0xc421a80000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc422d1ca80, 0xc4217ac1a0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc422d1ca80, 0xc4217ac1a0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc422d1ca80, 0xc4217ac1a0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420405970, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420405970) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 225 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420405970) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 290 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420405970) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 281 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc421cdb290, 0xc42136cd10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc421cdb290, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420215d68, 0xc422dacb90) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420215d68, 0xc422dacb90) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 282 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e120d9d0, 0x72, 0xc421372ce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc4210af098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc4210af098, 0xc421636000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc4210af080, 0xc421636000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc4210af080, 0xc421636000, 0x1000, 0x1000, 0xc42004e5d8, 0x1, 0xc42004e608) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc420215d68, 0xc421636000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc4219b7800, 0xc42157b230, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc4219b7800, 0xc42157b230, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc4219b7800, 0xc42157b230, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc421cdb290, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc421cdb290) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 283 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc421cdb290) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 284 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc421cdb290) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 285 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc423222480, 0xc421637000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).fill(0xc4216d7da0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).ReadByte(0xc4216d7da0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42274dce0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4227a6de0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4216d7e00, 0x16e4b60, 0xc4227a6e40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4216d7e00, 0x16e4b60, 0xc4227a6e40, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc421dc1580, 0x16e4b60, 0xc4227a6e40, 0xc4227a6e40, 0xc422dacc58) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc421dc1580, 0xc4227a6e40, 0x2d, 0xc42201ebe0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).readRequestHeader(0xc422dacc30, 0x1d72340, 0xc421dc1580, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42002b590, 0x8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).readRequest(0xc422dacc30, 0x1d72340, 0xc421dc1580, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc422dacc30, 0x1d72340, 0xc421dc1580, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d0e0, 0xc423222480, 0xc422dacc30) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 208 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e120d900, 0x72, 0xc422bd8858) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc42235ee18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc42235ee18, 0xc422b98000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc42235ee00, 0xc422b98000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc42235ee00, 0xc422b98000, 0x1000, 0x1000, 0xc420042500, 0xc422c2f3d8, 0xc422c2f3e0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc42000ff50, 0xc422b98000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).fill(0xc421c29e60) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).ReadByte(0xc421c29e60, 0x72, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4220d2900, 0xc42235ee18) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc422730720) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421c29ec0, 0x16e76e0, 0xc42065d780) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421c29ec0, 0x16e76e0, 0xc42065d780, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.decodeResponse(0xc422c54c80, 0x182ba60, 0xc422d9ec20, 0xc421996480, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:608 +0x5e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).genericRPC(0xc420404840, 0xc420355380, 0x24, 0xc4203625a0, 0x11, 0xa2c004821996400, 0x182ba00, 0xc421996480, 0x182ba60, 0xc422d9ec20, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:406 +0xf9 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).AppendEntries(0xc420404840, 0xc420355380, 0x24, 0xc4203625a0, 0x11, 0xc421996480, 0xc422d9ec20, 0x1, 0x3) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:379 +0x97 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicateTo(0xc4203b8000, 0xc423222600, 0x186416, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:186 +0x189 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4203b8000, 0xc423222600) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:132 +0x3f2 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203b8000, 0xc422730ae0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 207 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc420faeb40, 0xc422e5b700, 0x20, 0x20, 0x169d4e0, 0xc420619380) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422454ab0, 0x1d718c0, 0xc420faeb40, 0x0, 0xc420faeb00) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc42049d180, 0xc422e5b950, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4224191a0, 0xc42226c180, 0xc422454960, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.call(0xc420396480, 0xc4202156c8, 0x13, 0x1bda9a0, 0x4, 0xc422e5bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: reflect.Value.Call(0xc420396480, 0xc4202156c8, 0x13, 0xc422e5bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*service).call(0xc4225a6540, 0xc420218910, 0xc422762338, 0x0, 0xc420d9e800, 0xc422730a00, 0x19758e0, 0xc42226c180, 0x16, 0x169e020, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc420218910, 0x1d72340, 0xc422c55180, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d0e0, 0xc423222540, 0xc420218910) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 209 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc4203b8000, 0xc4232226c0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc4203b8000, 0xc4232226c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203b8000, 0xc422730b00) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 296 [runnable]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc42035c1e0, 0xc4205a4c60) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:179 +0xa8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc42049d180, 0xc42039fb01, 0xc42039fb80) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc42049d180, 0xc421e55740, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc42049d180, 0xc421e55740) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc4203e4b00, 0xc42049d180, 0xc421e55740) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 288 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4203b8000, 0xc423222600, 0xc422834e40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203b8000, 0xc422862700) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 297 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc4203b8000, 0xc4232226c0, 0xc421e557a0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203b8000, 0xc4227e6f40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 298 [runnable]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e120d760, 0x72, 0xc422e20ce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420453898, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420453898, 0xc422ba5000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc420453880, 0xc422ba5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc420453880, 0xc422ba5000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc42000fff0, 0xc422ba5000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc422583080, 0xc422762080, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc422583080, 0xc422762080, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc422583080, 0xc422762080, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422a57ce0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422a57ce0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 299 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422a57ce0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 300 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422a57ce0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 302 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).connect.func2(0x1d718c0, 0xc420185b60, 0xc420e7e400, 0xc421e55bc0, 0xc421e55aa0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:129 +0xde Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by net.(*netFD).connect Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:128 +0x286 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 289 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc422c2f580) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 322 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc4203b8000, 0xc4232226c0, 0x1d6eb80, 0xc422c2f580, 0xc422834f00, 0xc422834f60) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc4203b8000, 0xc422c2f5c0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 303 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc422a57ef0, 0xc422e1dd10, 0x30, 0xf9866a) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc422a57ef0, 0x1c61988, 0xc42049d180, 0x1d71840, 0xc420cfca40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420456d68, 0xc422805590) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d320, 0xc420456d68, 0xc422805590) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 304 [IO wait]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.runtime_pollWait(0x7f72e120d690, 0x72, 0xc42135dce8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).wait(0xc420e7e498, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*pollDesc).waitRead(0xc420e7e498, 0xc420b52000, 0x1000, 0x1000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: internal/poll.(*FD).Read(0xc420e7e480, 0xc420b52000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*netFD).Read(0xc420e7e480, 0xc420b52000, 0x1000, 0x1000, 0xc420fd65d8, 0x1, 0xc420fd6608) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net.(*conn).Read(0xc420456d68, 0xc420b52000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).Read(0xc420185c20, 0xc4224fadd0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadAtLeast(0x1d59aa0, 0xc420185c20, 0xc4224fadd0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: io.ReadFull(0x1d59aa0, 0xc420185c20, 0xc4224fadd0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc422a57ef0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc422a57ef0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 305 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc422a57ef0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 338 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc422a57ef0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 339 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420536cc0, 0xc420b53000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).fill(0xc420e8c960) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).ReadByte(0xc420e8c960, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422414db0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4227e7e40) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420e8c9c0, 0x16e4b60, 0xc4227e7ea0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420e8c9c0, 0x16e4b60, 0xc4227e7ea0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422be0c40, 0x16e4b60, 0xc4227e7ea0, 0xc4227e7ea0, 0xc422805658) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422be0c40, 0xc4227e7ea0, 0x2d, 0xc422574880) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).readRequestHeader(0xc422805630, 0x1d72340, 0xc422be0c40, 0x0, 0x0, 0x3, 0x100000000000003, 0xc42002b590, 0x8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).readRequest(0xc422805630, 0x1d72340, 0xc422be0c40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc422805630, 0x1d72340, 0xc422be0c40, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d0e0, 0xc420536cc0, 0xc422805630) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 341 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420c21ec0, 0xc42190d000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).fill(0xc420e8d020) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: bufio.(*Reader).ReadByte(0xc420e8d020, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc422415560, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42252e180) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420e8d080, 0x16e4b60, 0xc42252e1e0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420e8d080, 0x16e4b60, 0xc42252e1e0, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc422be1240, 0x16e4b60, 0xc42252e1e0, 0xc42252e1e0, 0xc4217967a8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc422be1240, 0xc42252e1e0, 0x2d, 0xc4228abf00) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).readRequestHeader(0xc421796780, 0x1d72340, 0xc422be1240, 0x30, 0x1, 0x3, 0x100000000000003, 0xc42002b590, 0x8) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).readRequest(0xc421796780, 0x1d72340, 0xc422be1240, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: net/rpc.(*Server).ServeRequest(0xc421796780, 0x1d72340, 0xc422be1240, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc42049d180, 0x1d71840, 0xc420cfca40, 0x1d7d0e0, 0xc420c21ec0, 0xc421796780) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 273 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc4204062d0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 354 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc4204062d0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 355 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc422455110) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 356 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422c04540, 0xc422e1ace8, 0x20, 0x20, 0x18, 0xc42252e4e0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422415bc0, 0x1d71840, 0xc422c04540, 0xc42252e4e0, 0x184987) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42039fb80, 0xc422e57ed0, 0x184987, 0x1d71840, 0xc422c04540, 0xc420bf51e0, 0xc4224550e0, 0xc4207f8150, 0x24, 0x2bba400) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420bf5180, 0x1d71840, 0xc422c04540, 0x184987, 0xc422be1600, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420bf5180, 0x1d71840, 0xc422c04540) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 357 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc422c046c0, 0xc422c1d770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc422875920, 0x1d71840, 0xc422c046c0, 0x0, 0x1849b5) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc42039fb80, 0xc422c1d960, 0x1849b5, 0x1d71840, 0xc422c046c0, 0x552bde, 0xc422c1d950, 0xc422c1d970, 0xf33ba5, 0xc420d69408) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc420d693b0, 0x1d71840, 0xc422c046c0, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc420d693b0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 358 [runnable]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: context.(*cancelCtx).Done(0xc422c04580, 0x1c664d0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /usr/local/go/src/context/context.go:325 +0xc8 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc4208c5860, 0x7f72e11b7588, 0xc422c04580, 0x1, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:252 +0x217 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc4208c5860, 0x7f72e11b7588, 0xc422c04580, 0x7f72e11b7588, 0xc422c04580) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc422c04700, 0x1833a4, 0x3b, 0xc422de6de8, 0x1, 0x1) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:214 +0x72 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc422c04700) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 342 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422415950, 0xc421e55f20) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 359 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc422c04840) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 360 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc420cf85a0, 0x1d71840, 0xc422c04580) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 343 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc422415950, 0xc421e55f80) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: goroutine 344 [select]: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).WaitN(0xc4208c5810, 0x7f72e11b7588, 0xc422be1780, 0x1, 0x0, 0x0) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:248 +0x2c1 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/vendor/golang.org/x/time/rate.(*Limiter).Wait(0xc4208c5810, 0x7f72e11b7588, 0xc422be1780, 0x7f72e11b7588, 0xc422be1780) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/golang.org/x/time/rate/rate.go:218 +0x48 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).latestEvalIndex(0xc4209ae000, 0xc422e1af58, 0xc422e1aef8, 0x4) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:784 +0x89 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).handleAllocUpdate(0xc4209ae000, 0xc4217cc008, 0x1, 0x1, 0xfb2095b3, 0x2b96de0, 0x2514b4606e, 0x50, 0xc422f56598, 0xf28f8d) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:514 +0x40 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc4209ae000) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:441 +0x591 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26161]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:29:50 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:29:50 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:29:50 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: ==> Starting Nomad agent... Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: ==> Nomad agent configuration: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: Client: false Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: Log Level: INFO Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: Region: amrv01 (DC: test) Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: Server: true Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: Version: 0.8.4 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:50 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:50 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:50 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:50 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:50.737676 [INFO] nomad: starting 4 scheduling worker(s) for [system service batch _core] Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:50 [INFO] serf: Attempting re-join to previously known node: ip-172-27-0-204.aav01: 172.27.0.204:4648 Aug 23 15:29:50 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:50.737988 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092004 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092028 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092041 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092053 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092065 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092076 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092087 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092269 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092284 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092313 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092326 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092337 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [WARN] memberlist: Refuting an alive message Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092636 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092654 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092666 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092677 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) 2018/08/23 15:29:51 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51 [INFO] serf: Re-joined to previously known node: ip-172-27-0-204.aav01: 172.27.0.204:4648 Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092771 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092782 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092793 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:29:51 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:51.092804 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [WARN] raft: Heartbeat timeout from "" reached, starting election Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 81 Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [INFO] raft: Duplicate RequestVote for same term: 81 Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [INFO] raft: Election won. Tally: 2 Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [INFO] raft: Node at 172.24.20.236:4647 [Leader] entering Leader state Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [INFO] raft: Added peer afff500f-c899-4d26-d003-1a0d1d8bf516, starting replication Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [INFO] raft: Added peer fa7f76f0-0581-92e0-d554-18ae538e5ae8, starting replication Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52.383295 [INFO] nomad: cluster leadership acquired Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [WARN] raft: AppendEntries to {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} rejected, sending older logs (next: 1598487) Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [INFO] raft: pipelining replication to peer {Voter fa7f76f0-0581-92e0-d554-18ae538e5ae8 172.24.10.89:4647} Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [INFO] raft: pipelining replication to peer {Voter afff500f-c899-4d26-d003-1a0d1d8bf516 172.24.0.121:4647} Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:52 ip-172-24-20-236 nomad.sh[26180]: 2018/08/23 15:29:52 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: fatal error: concurrent map writes Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 224 [running]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: runtime.throw(0x1bf7608, 0x15) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc42108e978 sp=0xc42108e958 pc=0x42d161 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: runtime.mapassign(0x1844be0, 0xc420503710, 0xc42108eaa8, 0x2bba400) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/hashmap.go:607 +0x553 fp=0xc42108ea08 sp=0xc42108e978 pc=0x40a403 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/lib.(*DelayHeap).Push(0xc4204739a0, 0x1d6dd40, 0xc42000f580, 0x2f30c22e, 0xed310877b, 0x0, 0x42bc79, 0xc400000008) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/lib/delay_heap.go:99 +0x1d1 fp=0xc42108eaf8 sp=0xc42108ea08 pc=0xf20d91 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*EvalBroker).processEnqueue(0xc420221860, 0xc42096b080, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:235 +0x26e fp=0xc42108eb60 sp=0xc42108eaf8 pc=0xf5388e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*EvalBroker).Enqueue(0xc420221860, 0xc42096b080) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:182 +0x83 fp=0xc42108eb98 sp=0xc42108eb60 pc=0xf534b3 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).restoreEvals(0xc420233400, 0xc420472e01, 0xc420472e40) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:299 +0x142 fp=0xc42108ec88 sp=0xc42108eb98 pc=0xf77602 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).establishLeadership(0xc420233400, 0xc420f3aa20, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:208 +0x2df fp=0xc42108ed38 sp=0xc42108ec88 pc=0xf76fff Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).leaderLoop(0xc420233400, 0xc420f3aa20) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:108 +0x510 fp=0xc42108efa0 sp=0xc42108ed38 pc=0xf76b90 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership.func1(0xc42091a1d0, 0xc420233400, 0xc420f3aa20) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:63 +0x5b fp=0xc42108efc8 sp=0xc42108efa0 pc=0xfb8e9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: runtime.goexit() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc42108efd0 sp=0xc42108efc8 pc=0x45c711 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:61 +0x1e2 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 1 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/command/agent.(*Command).handleSignals(0xc42030b810, 0xc4200ee780) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:638 +0x211 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/command/agent.(*Command).Run(0xc42030b810, 0xc42003a200, 0x1, 0x1, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/command.go:553 +0xdff Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc4202317c0, 0xc4202317c0, 0xc4203715c0, 0x37) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:255 +0x1eb Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: main.RunCustom(0xc42003a1f0, 0x2, 0x2, 0xc4204fff68) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:127 +0x4b4 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: main.Run(0xc42003a1f0, 0x2, 0x2, 0xc420090058) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:72 +0x3f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: main.main() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/main.go:68 +0x63 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 5 [syscall]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: os/signal.signal_recv(0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/sigqueue.go:139 +0xa6 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: os/signal.loop() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by os/signal.init.0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 38 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917bf00, 0x72, 0xc42057d9a8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc42035e098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc42035e098, 0xc42046c000, 0x1000, 0x1000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc42035e080, 0xc42046c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc42035e080, 0xc42046c000, 0x1000, 0x1000, 0x458600, 0xc4204a8c00, 0x4) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc42000e030, 0xc42046c000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/http.(*persistConn).Read(0xc420398000, 0xc42046c000, 0x1000, 0x1000, 0xc42057db98, 0x405675, 0xc420090900) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc4201f66c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).Peek(0xc4201f66c0, 0x1, 0x0, 0x0, 0x1, 0xc420f97860, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/http.(*persistConn).readLoop(0xc420398000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by net/http.(*Transport).dialConn Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 26 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*RaftLayer).Accept(0xc420201220, 0x1c636a0, 0xc4202debb0, 0x1d71840, 0xc4203e48c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/raft_rpc.go:63 +0xe1 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen(0xc4202debb0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:466 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.NewNetworkTransportWithConfig Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:166 +0x18a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 69 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42027e0b0, 0xc420605d10, 0x30, 0xf9866a) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42027e0b0, 0x1c61988, 0xc420233400, 0x1d71840, 0xc4213725c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc4202040b0, 0xc4212c8190) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc4202040b0, 0xc4212c8190) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 9 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*InmemSignal).run(0xc4203e4300) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:64 +0xe4 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.NewInmemSignal Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/inmem_signal.go:38 +0x150 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 57 [select, locked to thread]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: runtime.gopark(0x1c661d0, 0x0, 0x1bde186, 0x6, 0x18, 0x1) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/proc.go:291 +0x11a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: runtime.selectgo(0xc4204ad750, 0xc420090660) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/select.go:392 +0xe50 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: runtime.ensureSigM.func1() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/signal_unix.go:549 +0x1f4 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: runtime.goexit() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 10 [sleep]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: time.Sleep(0x3b9aca00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/time.go:102 +0x166 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.(*Metrics).collectStats(0xc42022eb00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/metrics.go:168 +0x2f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics.New Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/armon/go-metrics/start.go:75 +0x121 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 11 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run(0xc420228480) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:320 +0x4b4 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/command/agent.(*Agent).setupConsul Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/agent.go:971 +0x175 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 12 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/helper/pool.(*ConnPool).reap(0xc4202f4de0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:439 +0x325 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/helper/pool.NewPool Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/helper/pool/pool.go:170 +0x11f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 16 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/command/agent/consul.(*checkWatcher).Run(0xc4204a4dc0, 0x1d71840, 0xc4203e4340) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/check_watcher.go:186 +0x2b8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/command/agent/consul.(*ServiceClient).Run Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/consul/client.go:313 +0x311 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 91 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*serfQueries).stream(0xc421361020) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:80 +0xe0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.newSerfQueries Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/internal_query.go:73 +0xbc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 14 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*vaultClient).revokeDaemon(0xc420560340) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1114 +0x168 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*vaultClient).(github.com/hashicorp/nomad/nomad.revokeDaemon)-fm() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:248 +0x2a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.wrapNilError.func1(0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1186 +0x24 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).run(0xc420503b60, 0xc4202113d0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:153 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/gopkg.in/tomb%2ev2.(*Tomb).Go Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/gopkg.in/tomb.v2/tomb.go:149 +0xb9 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 90 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runSnapshots(0xc420192000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/snapshot.go:71 +0x152 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runSnapshots)-fm() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:507 +0x2a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420192000, 0xc42053b4a0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 39 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/http.(*persistConn).writeLoop(0xc420398000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by net/http.(*Transport).dialConn Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 142 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/http.(*persistConn).writeLoop(0xc4205fa000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/transport.go:1822 +0x14b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by net/http.(*Transport).dialConn Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/transport.go:1238 +0x97f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 68 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202dedc0, 0xc4212ffd10, 0x30, 0xf9866a) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202dedc0, 0x1c61988, 0xc420233400, 0x1d71840, 0xc4213725c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc420204048, 0xc4212c80f0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc420204048, 0xc4212c80f0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 93 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917bd60, 0x72, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc420f44018, 0x72, 0xc4203f5c00, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc420f44018, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Accept(0xc420f44000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).accept(0xc420f44000, 0x0, 0xc420f87620, 0xa10ac6) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*TCPListener).accept(0xc42035c180, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*TCPListener).AcceptTCP(0xc42035c180, 0xa10b06, 0x1, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/tcpsock.go:246 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).tcpListen(0xc4205be3c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:204 +0x33 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:144 +0x714 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 70 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917ba20, 0x72, 0xc4212fece8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc420e8e198, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc420e8e198, 0xc42079a000, 0x1000, 0x1000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc420e8e180, 0xc42079a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc420e8e180, 0xc42079a000, 0x1000, 0x1000, 0xa8, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc4202040b0, 0xc42079a000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).Read(0xc4212ce1e0, 0xc42130c0b0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadAtLeast(0x1d59aa0, 0xc4212ce1e0, 0xc42130c0b0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadFull(0x1d59aa0, 0xc4212ce1e0, 0xc42130c0b0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42027e0b0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42027e0b0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 71 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42027e0b0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 89 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runFSM(0xc420192000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/fsm.go:116 +0x1e7 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.runFSM)-fm() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:506 +0x2a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420192000, 0xc42053b490) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 92 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Snapshotter).stream(0xc42097e410) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:180 +0x1a4 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.NewSnapshotter Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/snapshot.go:125 +0x550 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 96 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4205be3c0, 0x12a05f200, 0xc42137df80, 0xc420b4dda0, 0xc42053bae0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:82 +0x373 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 97 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullTrigger(0xc4205be3c0, 0xc420b4dda0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:134 +0x110 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:88 +0x2cc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 146 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc(0xc4205be3c0, 0x1dcd6500, 0xc421006000, 0xc420b4dda0, 0xc42053bb00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:116 +0x1a8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:94 +0x1c2 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 183 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420228840, 0xc420e17000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc421318660) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc421318660, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d1da10, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42095b3e0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4213186c0, 0x16e4b60, 0xc42095b440) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4213186c0, 0x16e4b60, 0xc42095b440, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420083fc0, 0x16e4b60, 0xc42095b440, 0xc42095b440, 0xc4212c8258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420083fc0, 0xc42095b440, 0x2d, 0xc422287080) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 systemd[1]: nomad.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc4212c8230, 0x1d72340, 0xc420083fc0, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc4212c8230, 0x1d72340, 0xc420083fc0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc4212c8230, 0x1d72340, 0xc420083fc0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420228840, 0xc4212c8230) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 95 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpHandler(0xc4205be3c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:400 +0x109 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:146 +0x75e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 88 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).leaderLoop(0xc420192000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:500 +0x415 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).runLeader(0xc420192000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:420 +0x385 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).run(0xc420192000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:140 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).(github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.run)-fm() Aug 23 15:29:53 ip-172-24-20-236 systemd[1]: nomad.service: Unit entered failed state. Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/api.go:505 +0x2a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420192000, 0xc42053b480) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 134 [runnable]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: syscall.Syscall(0x0, 0xa, 0xc420fa0000, 0x1000, 0xffffffffffffffff, 0x0, 0xb) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: syscall.read(0xa, 0xc420fa0000, 0x1000, 0x1000, 0xc422a39d00, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/syscall/zsyscall_linux_amd64.go:749 +0x5f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: syscall.Read(0xa, 0xc420fa0000, 0x1000, 0x1000, 0x9c, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/syscall/syscall_unix.go:162 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc420e8e100, 0xc420fa0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:153 +0x118 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc420e8e100, 0xc420fa0000, 0x1000, 0x1000, 0x1c60d60, 0xc420c68b78, 0xc420c68b68) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc420204048, 0xc420fa0000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).Read(0xc421397500, 0xc420b0d500, 0x97, 0x304, 0x1b70ac0, 0x7fb6c917bff0, 0xc420c68be7) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.(*LimitedReader).Read(0xc420018420, 0xc420b0d500, 0x304, 0x304, 0xc420b56c00, 0xc420040000, 0xc420c68c68) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:446 +0x63 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bytes.(*Buffer).ReadFrom(0xc420fcefc0, 0x1d5bf40, 0xc420018420, 0x7fb6c917bff0, 0xc420fcefc0, 0xc420c68c01) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bytes/buffer.go:205 +0xa0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.copyBuffer(0x1d59b20, 0xc420fcefc0, 0x1d5bf40, 0xc420018420, 0x0, 0x0, 0x0, 0x1944e20, 0x1, 0xc420018420) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:386 +0x31a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.Copy(0x1d59b20, 0xc420fcefc0, 0x1d5bf40, 0xc420018420, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:362 +0x5a Aug 23 15:29:53 ip-172-24-20-236 systemd[1]: nomad.service: Failed with result 'exit-code'. Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).readData(0xc420ad6480, 0xc42130c0e0, 0xc, 0xc, 0x0, 0x1d59aa0, 0xc421397500, 0xc420c68eb0, 0x466056) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:423 +0x148 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).handleStreamMessage(0xc4202dedc0, 0xc42130c0e0, 0xc, 0xc, 0xc, 0xc) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:518 +0x29b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202dedc0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:469 +0x176 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202dedc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 214 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42044aa80, 0xc420d77000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420cde540) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc420cde540, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420c51cb0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421383260) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420cde5a0, 0x16e4b60, 0xc4213832c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420cde5a0, 0x16e4b60, 0xc4213832c0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4207dd180, 0x16e4b60, 0xc4213832c0, 0xc4213832c0, 0xc421334578) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4207dd180, 0xc4213832c0, 0x2d, 0xc420dbcc40) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc421334550, 0x1d72340, 0xc4207dd180, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc421334550, 0x1d72340, 0xc4207dd180, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc421334550, 0x1d72340, 0xc4207dd180, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc42044aa80, 0xc421334550) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 260 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42323c180, 0xc421015d28, 0x20, 0x20, 0xc4202e6c00, 0x1bd9a98) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4230eaed0, 0x1d71840, 0xc42323c180, 0xc4230fa3e0, 0x1849b5) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420472e40, 0xc420c9ff20, 0x1849b5, 0x1d71840, 0xc42323c180, 0x1d55e50, 0x0, 0x0, 0x7fb6a7023536, 0x299) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocs(0xc423088140, 0x1849b5, 0x63c8dd, 0x1, 0x0, 0x0, 0x0, 0xc4210b8060) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:743 +0x70 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh.func1(0xc423088140, 0x1849b5, 0xc4225c6900) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:729 +0x42 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).getAllocsCh Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:728 +0x71 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 94 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917bc90, 0x72, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc420f44098, 0x72, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc420f44098, 0xc423366000, 0x10000, 0x10000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).ReadFrom(0xc420f44080, 0xc423366000, 0x10000, 0x10000, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:207 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).readFrom(0xc420f44080, 0xc423366000, 0x10000, 0x10000, 0x2a, 0x2a, 0x1d62b40, 0x412539, 0xc42102fe40) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:208 +0x5b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*UDPConn).readFrom(0xc42035c1a0, 0xc423366000, 0x10000, 0x10000, 0x101, 0xc42102fe18, 0xc42102fe90, 0x443cf7) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*UDPConn).ReadFrom(0xc42035c1a0, 0xc423366000, 0x10000, 0x10000, 0x10000, 0x10000, 0x525ee, 0xbbc5c472, 0x2b96de0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/udpsock.go:118 +0x6f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).udpListen(0xc4205be3c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:302 +0x12b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.newMemberlist Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:145 +0x739 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 147 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReap(0xc4204a9980) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1412 +0x154 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:406 +0x956 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 148 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).handleReconnect(0xc4204a9980) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1430 +0x101 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:407 +0x97b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 149 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204a9980, 0x1bdd09a, 0x6, 0xc421361170) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:408 +0x9be Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 150 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204a9980, 0x1bdba6c, 0x5, 0xc4213611a0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:409 +0xa04 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 151 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).checkQueueDepth(0xc4204a9980, 0x1bdbc6a, 0x5, 0xc4213611d0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:1545 +0x103 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.Create Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:410 +0xa4a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 187 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917b540, 0x72, 0xc421017ce8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc42096f118, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc42096f118, 0xc420c32000, 0x1000, 0x1000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc42096f100, 0xc420c32000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc42096f100, 0xc420c32000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc42000f598, 0xc420c32000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).Read(0xc421006b40, 0xc4206e0fa0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadAtLeast(0x1d59aa0, 0xc421006b40, 0xc4206e0fa0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadFull(0x1d59aa0, 0xc421006b40, 0xc4206e0fa0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc420f2cdc0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc420f2cdc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 153 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421006060, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421006060, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421006060) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 154 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc4210060c0, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc4210060c0, 0x1dcd6500, 0x1bd9928, 0x2, 0x1bdab30, 0x4, 0xc4207f0200) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc4210060c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 155 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421006120, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421006120, 0x1dcd6500, 0x0, 0x0, 0x0, 0x435248, 0x1c66060) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421006120) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 156 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).backoffErr(0xc421006180, 0x1312d00, 0x2540be400, 0x19) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:527 +0x122 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).dequeueEvaluation(0xc421006180, 0x1dcd6500, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:173 +0x477 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Worker).run(0xc421006180) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:109 +0x38 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewWorker Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/worker.go:82 +0x119 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 157 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917b950, 0x72, 0xc420f67490) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc4228e8218, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc4228e8218, 0xc42083b000, 0x1000, 0x1000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc4228e8200, 0xc42083b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc4228e8200, 0xc42083b000, 0x1000, 0x1000, 0x1, 0xc42083b000, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc42000e668, 0xc42083b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).Read(0xc422a3de60, 0xc4233221a8, 0x1, 0x1, 0x1000, 0x1000, 0xc420f67698) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).readTCP(0xc4205be3c0, 0x1d7d320, 0xc42000e668, 0x2b96de0, 0x0, 0x0, 0x2b96de0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:912 +0xee Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).sendAndReceiveState(0xc4205be3c0, 0xc422d6abf0, 0x10, 0x10, 0x11228, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/net.go:772 +0x357 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).pushPullNode(0xc4205be3c0, 0xc422d6abf0, 0x10, 0x10, 0x11228, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/state.go:546 +0x14f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist.(*Memberlist).Join(0xc4205be3c0, 0xc420cae000, 0x14, 0x20, 0x3, 0x3, 0xc420a22c20) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/memberlist/memberlist.go:190 +0x128 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf.(*Serf).Join(0xc4204a9980, 0xc420cae000, 0x14, 0x20, 0xc420a22c01, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/serf/serf/serf.go:604 +0xdd Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).Join(0xc420233400, 0xc420cae000, 0x14, 0x20, 0xc420a22c20, 0x10, 0xc420b90b40) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:1326 +0x55 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func2(0xc4204b0740, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:840 +0xb87 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler.func3(0xc420b17230, 0xc420233400) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:860 +0x131 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).setupBootstrapHandler Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:853 +0x110 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 158 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).monitorLeadership(0xc420233400) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:50 +0x105 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:374 +0x139d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 159 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).serfEventHandler(0xc420233400) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/serf.go:28 +0xe7 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:377 +0x13c2 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 160 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917be30, 0x72, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc420135918, 0x72, 0xc4204dd300, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc420135918, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Accept(0xc420135900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).accept(0xc420135900, 0x1, 0x1, 0xc420604f20) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*TCPListener).accept(0xc42035cb68, 0x406b4a, 0xc420f962a0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*TCPListener).Accept(0xc42035cb68, 0xc420f962a0, 0x0, 0x3f800000, 0xc4213725c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).listen(0xc420233400, 0x1d71840, 0xc4213725c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:77 +0xae Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).startRPCListener Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:408 +0x92 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 161 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*EvalBroker).EmitStats(0xc420221860, 0x3b9aca00, 0xc4202e0900) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:818 +0x2c6 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:383 +0x1415 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 162 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*PlanQueue).EmitStats(0xc420397540, 0x3b9aca00, 0xc4202e0900) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:200 +0x10d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:386 +0x1457 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 163 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).EmitStats(0xc42024a630, 0x3b9aca00, 0xc4202e0900) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:601 +0x10d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:389 +0x1499 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 164 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*vaultClient).EmitStats(0xc420560340, 0x3b9aca00, 0xc4202e0900) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/vault.go:1215 +0x10d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:392 +0x14e0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 165 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).heartbeatStats(0xc420233400) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/heartbeat.go:177 +0x11d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewServer Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/server.go:395 +0x1505 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 166 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917baf0, 0x72, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc42096e818, 0x72, 0xc4204a4600, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc42096e818, 0xffffffffffffff00, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Accept(0xc42096e800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:372 +0x1a8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).accept(0xc42096e800, 0x402037, 0xc420353980, 0x1805ec0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:238 +0x42 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*TCPListener).accept(0xc42035cb88, 0x458e40, 0xc420580e90, 0xc420580e98) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*TCPListener).Accept(0xc42035cb88, 0x1c65af8, 0xc420353900, 0x1d71900, 0xc42134f3e0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/tcpsock.go:259 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/http.(*Server).Serve(0xc42098f1e0, 0x1d6fc00, 0xc42035cb88, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/server.go:2770 +0x1a5 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/http.Serve(0x1d6fc00, 0xc42035cb88, 0x1d5df00, 0xc42134f320, 0x3339663037352400, 0x2d626239392d3466) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/server.go:2389 +0x73 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/command/agent.NewHTTPServer.func1(0xc421372680, 0x1d6fc00, 0xc42035cb88, 0xc4205160d0, 0xc420b172f0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:107 +0x82 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/command/agent.NewHTTPServer Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/command/agent/http.go:105 +0x3c2 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 167 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval.func1(0x4e94914f0000, 0xc42096e880, 0xc4205160e0, 0xc420b4dec0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:314 +0x120 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint.CheckInterval Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-checkpoint/checkpoint.go:312 +0xbb Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 72 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42027e0b0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 135 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202dedc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 136 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202dedc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 73 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42044a600, 0xc42079b000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420086ea0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc420086ea0, 0x101000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42134ef30, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420fe20c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420086f00, 0x16e4b60, 0xc420fe2120) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420086f00, 0x16e4b60, 0xc420fe2120, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e82500, 0x16e4b60, 0xc420fe2120, 0xc420fe2120, 0xc4212c8258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e82500, 0xc420fe2120, 0x2d, 0xc420670720) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc4212c8230, 0x1d72340, 0xc420e82500, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc4212c8230, 0x1d72340, 0xc420e82500, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc4212c8230, 0x1d72340, 0xc420e82500, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc42044a600, 0xc4212c8230) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 137 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc42044a6c0, 0xc420fa1000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420186a80) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc420186a80, 0x101000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42136f6e0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc4213823c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420186ae0, 0x16e4b60, 0xc421382420) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420186ae0, 0x16e4b60, 0xc421382420, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420c1ef80, 0x16e4b60, 0xc421382420, 0xc421382420, 0xc421334578) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420c1ef80, 0xc421382420, 0x2d, 0xc421564360) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc421334550, 0x1d72340, 0xc420c1ef80, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc421334550, 0x1d72340, 0xc420c1ef80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc421334550, 0x1d72340, 0xc420c1ef80, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc42044a6c0, 0xc421334550) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 76 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42027e160, 0xc420c64d10, 0x30, 0xf9866a) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42027e160, 0x1c61988, 0xc420233400, 0x1d71840, 0xc4213725c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc420204f18, 0xc4212c8500) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc420204f18, 0xc4212c8500) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 144 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc4202dee70, 0xc420c6bd10, 0x30, 0xf9866a) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc4202dee70, 0x1c61988, 0xc420233400, 0x1d71840, 0xc4213725c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc42000ed58, 0xc4213349b0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc42000ed58, 0xc4213349b0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 141 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917b880, 0x72, 0xc420c6a9a8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc420f0ef18, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc420f0ef18, 0xc4203c8000, 0x1000, 0x1000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc420f0ef00, 0xc4203c8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc420f0ef00, 0xc4203c8000, 0x1000, 0x1000, 0x458600, 0xc420b57980, 0x4) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc42000ed30, 0xc4203c8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/http.(*persistConn).Read(0xc4205fa000, 0xc4203c8000, 0x1000, 0x1000, 0xc420c6ab98, 0x405675, 0xc420f1c9c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/transport.go:1453 +0x136 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420187260) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).Peek(0xc420187260, 0x1, 0x0, 0x0, 0x1, 0xc420e0df20, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:132 +0x3a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/http.(*persistConn).readLoop(0xc4205fa000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/transport.go:1601 +0x185 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by net/http.(*Transport).dialConn Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/http/transport.go:1237 +0x95a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 179 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4202280c0, 0xc4209f8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420f63320) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc420f63320, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420d1c360, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42095a4e0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420f63380, 0x16e4b60, 0xc42095a540) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420f63380, 0x16e4b60, 0xc42095a540, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420c47b40, 0x16e4b60, 0xc42095a540, 0xc42095a540, 0xc4212c8988) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420c47b40, 0xc42095a540, 0x2d, 0xc42088f1e0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc4212c8960, 0x1d72340, 0xc420c47b40, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc4212c8960, 0x1d72340, 0xc420c47b40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc4212c8960, 0x1d72340, 0xc420c47b40, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc4202280c0, 0xc4212c8960) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 77 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917b7b0, 0x72, 0xc420c98ce8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc420f94918, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc420f94918, 0xc420842000, 0x1000, 0x1000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc420f94900, 0xc420842000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc420f94900, 0xc420842000, 0x1000, 0x1000, 0xc4212f45d8, 0x1, 0xc4212f4608) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc420204f18, 0xc420842000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).Read(0xc4200877a0, 0xc420c4b710, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadAtLeast(0x1d59aa0, 0xc4200877a0, 0xc420c4b710, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadFull(0x1d59aa0, 0xc4200877a0, 0xc420c4b710, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42027e160, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42027e160) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 78 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42027e160) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 79 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42027e160) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 80 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420174240, 0xc420843000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc4207c6c00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc4207c6c00, 0x101000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4208f46c0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420fe25e0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4207c6c60, 0x16e4b60, 0xc420fe2640) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4207c6c60, 0x16e4b60, 0xc420fe2640, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e82d40, 0x16e4b60, 0xc420fe2640, 0xc420fe2640, 0xc4212c85c8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e82d40, 0xc420fe2640, 0x2d, 0xc42269b100) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc4212c85a0, 0x1d72340, 0xc420e82d40, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc4212c85a0, 0x1d72340, 0xc420e82d40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc4212c85a0, 0x1d72340, 0xc420e82d40, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420174240, 0xc4212c85a0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 281 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*deploymentWatcher).watch(0xc423088140) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:387 +0x304 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.newDeploymentWatcher Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployment_watcher.go:121 +0x189 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 280 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc423208000, 0xc423218060) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 196 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42027e210, 0xc420c67d10, 0x30, 0xf9866a) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42027e210, 0x1c61988, 0xc420233400, 0x1d71840, 0xc4213725c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc420205278, 0xc4212c88c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc420205278, 0xc4212c88c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 197 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917b6e0, 0x72, 0xc4212f8ce8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc420a27818, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc420a27818, 0xc420723000, 0x1000, 0x1000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc420a27800, 0xc420723000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc420a27800, 0xc420723000, 0x1000, 0x1000, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc420205278, 0xc420723000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).Read(0xc4207c7740, 0xc42091aab0, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadAtLeast(0x1d59aa0, 0xc4207c7740, 0xc42091aab0, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadFull(0x1d59aa0, 0xc4207c7740, 0xc42091aab0, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42027e210, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42027e210) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 198 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42027e210) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 199 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42027e210) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 145 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917b610, 0x72, 0xc420c94ce8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc420f0f098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc420f0f098, 0xc420a38000, 0x1000, 0x1000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc420f0f080, 0xc420a38000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc420f0f080, 0xc420a38000, 0x1000, 0x1000, 0xc42004f5d8, 0x1, 0xc42004f608) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc42000ed58, 0xc420a38000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).Read(0xc4201876e0, 0xc42133cb00, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadAtLeast(0x1d59aa0, 0xc4201876e0, 0xc42133cb00, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadFull(0x1d59aa0, 0xc4201876e0, 0xc42133cb00, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc4202dee70, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc4202dee70) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 210 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc4202dee70) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 211 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc4202dee70) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 212 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420ad6240, 0xc420a39000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420af5b00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc420af5b00, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420c510b0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc421382bc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420af5b60, 0x16e4b60, 0xc421382c20) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420af5b60, 0x16e4b60, 0xc421382c20, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420aba300, 0x16e4b60, 0xc421382c20, 0xc421382c20, 0xc421334a78) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420aba300, 0xc421382c20, 0x2d, 0xc420dbcf80) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc421334a50, 0x1d72340, 0xc420aba300, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc421334a50, 0x1d72340, 0xc420aba300, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc421334a50, 0x1d72340, 0xc420aba300, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420ad6240, 0xc421334a50) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 200 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420ad6300, 0xc420da2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420827ce0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc420827ce0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a837a0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420fe2c20) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420827d40, 0x16e4b60, 0xc420fe2c80) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420827d40, 0x16e4b60, 0xc420fe2c80, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e83800, 0x16e4b60, 0xc420fe2c80, 0xc420fe2c80, 0xc4212c8258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e83800, 0xc420fe2c80, 0x2d, 0xc422286360) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc4212c8230, 0x1d72340, 0xc420e83800, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc4212c8230, 0x1d72340, 0xc420e83800, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc4212c8230, 0x1d72340, 0xc420e83800, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420ad6300, 0xc4212c8230) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 201 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420ad63c0, 0xc420dc2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420dbe060) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc420dbe060, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a83a40, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420fe2da0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420dbe0c0, 0x16e4b60, 0xc420fe2e00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420dbe0c0, 0x16e4b60, 0xc420fe2e00, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e83940, 0x16e4b60, 0xc420fe2e00, 0xc420fe2e00, 0xc4212c8258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e83940, 0xc420fe2e00, 0x2d, 0xc420dbd380) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc4212c8230, 0x1d72340, 0xc420e83940, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc4212c8230, 0x1d72340, 0xc420e83940, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc4212c8230, 0x1d72340, 0xc420e83940, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420ad63c0, 0xc4212c8230) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 185 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420228e40, 0xc420982000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc421318fc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc421318fc0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc42085ca20, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42095bfa0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc421319020, 0x16e4b60, 0xc420b90000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc421319020, 0x16e4b60, 0xc420b90000, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4203da740, 0x16e4b60, 0xc420b90000, 0xc420b90000, 0xc421334578) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4203da740, 0xc420b90000, 0x2d, 0xc420dbcec0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc421334550, 0x1d72340, 0xc4203da740, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc421334550, 0x1d72340, 0xc4203da740, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc421334550, 0x1d72340, 0xc4203da740, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420228e40, 0xc421334550) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 202 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420ad6480, 0xc420e24000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420dbe420) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc420dbe420, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420a83cb0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420fe2f00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420dbe480, 0x16e4b60, 0xc420fe2f60) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420dbe480, 0x16e4b60, 0xc420fe2f60, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e83a00, 0x16e4b60, 0xc420fe2f60, 0xc420fe2f60, 0xc421334578) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e83a00, 0xc420fe2f60, 0x2d, 0xc423258920) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc421334550, 0x1d72340, 0xc420e83a00, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc421334550, 0x1d72340, 0xc420e83a00, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc421334550, 0x1d72340, 0xc420e83a00, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420ad6480, 0xc421334550) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 204 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420ad6780, 0xc421002000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420dbf8c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc420dbf8c0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4207eb560, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc420fe3940) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420dbf920, 0x16e4b60, 0xc420fe39a0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420dbf920, 0x16e4b60, 0xc420fe39a0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420e83ec0, 0x16e4b60, 0xc420fe39a0, 0xc420fe39a0, 0xc421334578) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420e83ec0, 0xc420fe39a0, 0x2d, 0xc420dbc880) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc421334550, 0x1d72340, 0xc420e83ec0, 0x60, 0x2, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc421334550, 0x1d72340, 0xc420e83ec0, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc421334550, 0x1d72340, 0xc420e83ec0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420ad6780, 0xc421334550) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 209 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc420192000, 0xc420ad69c0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc420192000, 0xc420ad69c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420192000, 0xc42095aae0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 219 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc420f2cdc0, 0xc4212fbd10, 0x30, 0xf9866a) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc420f2cdc0, 0x1c61988, 0xc420233400, 0x1d71840, 0xc4213725c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc42000f598, 0xc420185450) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc42000f598, 0xc420185450) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 188 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc420f2cdc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 189 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc420f2cdc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 190 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc42137daa0, 0xc420ca3700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc420e7cf00, 0x1d718c0, 0xc42137daa0, 0x0, 0xc42137da00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420233400, 0xc420ca3950, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc420c34900, 0xc420f44380, 0xc420e7ce10, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: reflect.Value.call(0xc42037e960, 0xc42035cfc8, 0x13, 0x1bda9a0, 0x4, 0xc420ca3d48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: reflect.Value.Call(0xc42037e960, 0xc42035cfc8, 0x13, 0xc420ca3d48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*service).call(0xc4209a5400, 0xc4205ea190, 0xc420f14e58, 0x0, 0xc42070de80, 0xc420b910c0, 0x19758e0, 0xc420f44380, 0x16, 0x169e020, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc4205ea190, 0x1d72340, 0xc4209a54c0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420174300, 0xc4205ea190) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 171 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420ad6900, 0xc4207c2000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc420b9dda0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc420b9dda0, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420f7c8a0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42076cd40) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc420b9de00, 0x16e4b60, 0xc42076cda0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc420b9de00, 0x16e4b60, 0xc42076cda0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc420acbf80, 0x16e4b60, 0xc42076cda0, 0xc42076cda0, 0xc4212c8258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc420acbf80, 0xc42076cda0, 0x2d, 0xc422286fa0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc4212c8230, 0x1d72340, 0xc420acbf80, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc4212c8230, 0x1d72340, 0xc420acbf80, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc4212c8230, 0x1d72340, 0xc420acbf80, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420ad6900, 0xc4212c8230) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 208 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate(0xc420192000, 0xc420ad66c0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:393 +0x450 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate(0xc420192000, 0xc420ad66c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:152 +0x299 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).startStopReplication.func1() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/raft.go:451 +0x33 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420192000, 0xc42095aac0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 206 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc420ad6540, 0xc420e89000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc421397860) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc421397860, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc420f7e000, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42095a240) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4213978c0, 0x16e4b60, 0xc42095a2a0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4213978c0, 0x16e4b60, 0xc42095a2a0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4209a5a40, 0x16e4b60, 0xc42095a2a0, 0xc42095a2a0, 0xc4212c8258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4209a5a40, 0xc42095a2a0, 0x2d, 0xc42088fea0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc4212c8230, 0x1d72340, 0xc4209a5a40, 0x30, 0x1, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc4212c8230, 0x1d72340, 0xc4209a5a40, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc4212c8230, 0x1d72340, 0xc4209a5a40, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420ad6540, 0xc4212c8230) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 279 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*EvaluatePool).run(0xc423208000, 0xc423218000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:104 +0x1d1 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.NewEvaluatePool Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply_pool.go:49 +0x13b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 246 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).run(0xc420a18640) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:108 +0x19d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:80 +0x114 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 207 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917b200, 0x72, 0xc42120fac8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc420f0ef98, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc420f0ef98, 0xc420b48000, 0x1000, 0x1000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc420f0ef80, 0xc420b48000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc420f0ef80, 0xc420b48000, 0x1000, 0x1000, 0xc4207f62a0, 0xc42131b1f0, 0xc42030c2a0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc42035c298, 0xc420b48000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc42037ede0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc42037ede0, 0x3, 0xc420040070, 0xc420040000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleCommand(0xc4202debb0, 0xc42037ede0, 0xc42037ee40, 0xc42030c2a0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:515 +0x43 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).handleConn(0xc4202debb0, 0x1d71840, 0xc4203e48c0, 0x1d7d320, 0xc42035c298) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:499 +0x23d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*NetworkTransport).listen Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:477 +0x1ce Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 226 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).AcceptStream(0xc42027f130, 0xc421213d10, 0x30, 0xf9866a) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:207 +0xe8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).Accept(0xc42027f130, 0x1c61988, 0xc420233400, 0x1d71840, 0xc4213725c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:197 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc4202055e0, 0xc421335680) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:211 +0x2b8 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d320, 0xc4202055e0, 0xc421335680) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:137 +0x2ef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).listen Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:93 +0x15b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 176 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc420192000, 0xc420ad66c0, 0xc420f97860) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420192000, 0xc421383cc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 225 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).heartbeat(0xc420192000, 0xc420ad69c0, 0xc420f3ac00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:340 +0x2ae Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).replicate.func1() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:118 +0x3c Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420192000, 0xc420b90c80) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 227 [IO wait]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.runtime_pollWait(0x7fb6c917b130, 0x72, 0xc42148fce8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/netpoll.go:173 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).wait(0xc420af8098, 0x72, 0xffffffffffffff00, 0x1d5e4e0, 0x2ade258) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*pollDesc).waitRead(0xc420af8098, 0xc420948000, 0x1000, 0x1000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: internal/poll.(*FD).Read(0xc420af8080, 0xc420948000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/internal/poll/fd_unix.go:157 +0x17d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*netFD).Read(0xc420af8080, 0xc420948000, 0x1000, 0x1000, 0xfc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/fd_unix.go:202 +0x4f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net.(*conn).Read(0xc4202055e0, 0xc420948000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/net.go:176 +0x6a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).Read(0xc42037f800, 0xc42148a730, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:216 +0x238 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadAtLeast(0x1d59aa0, 0xc42037f800, 0xc42148a730, 0xc, 0xc, 0xc, 0xc, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:309 +0x86 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: io.ReadFull(0x1d59aa0, 0xc42037f800, 0xc42148a730, 0xc, 0xc, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/io/io.go:327 +0x58 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recvLoop(0xc42027f130, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:451 +0xef Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).recv(0xc42027f130) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:430 +0x2b Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:108 +0x34d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 228 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).send(0xc42027f130) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:392 +0x2a0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:109 +0x36f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 229 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Session).keepalive(0xc42027f130) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:308 +0xee Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.newSession Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/session.go:111 +0x3be Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 230 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux.(*Stream).Read(0xc4202286c0, 0xc420949000, 0x1000, 0x1000, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/yamux/stream.go:133 +0x301 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).fill(0xc4212cec60) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:100 +0x11e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: bufio.(*Reader).ReadByte(0xc4212cec60, 0x100000000000016, 0x0, 0x2d) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/bufio/bufio.go:242 +0x39 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*ioDecReader).readn1(0xc4203e1200, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:90 +0x7a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*msgpackDecDriver).initReadNext(0xc42095b0c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/msgpack.go:540 +0x45 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).decode(0xc4212cecc0, 0x16e4b60, 0xc42095b140) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:635 +0x49 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec.(*Decoder).Decode(0xc4212cecc0, 0x16e4b60, 0xc42095b140, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-msgpack/codec/decode.go:630 +0x6d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).read(0xc4204dd880, 0x16e4b60, 0xc42095b140, 0xc42095b140, 0xc421335748) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:121 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc.(*MsgpackCodec).ReadRequestHeader(0xc4204dd880, 0xc42095b140, 0x2d, 0xc420dbcb60) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/net-rpc-msgpackrpc/codec.go:60 +0x41 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequestHeader(0xc421335720, 0x1d72340, 0xc4204dd880, 0x0, 0x0, 0x3, 0x100000000000003, 0xc420184f50, 0x8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:589 +0x67 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).readRequest(0xc421335720, 0x1d72340, 0xc4204dd880, 0x3, 0x3, 0x6, 0x3f800000, 0x0, 0x1, 0x1, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:549 +0x5a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc421335720, 0x1d72340, 0xc4204dd880, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:492 +0x74 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc4202286c0, 0xc421335720) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 192 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d718c0, 0xc4212cee40, 0xc420f6b700, 0x20, 0x20, 0x30, 0x30) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4203d6e10, 0x1d718c0, 0xc4212cee40, 0x0, 0xc4212cee00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).blockingRPC(0xc420233400, 0xc420f6b950, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:662 +0x276 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Node).GetClientAllocs(0xc4212ce720, 0xc420d1b900, 0xc4203d6a80, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/node_endpoint.go:980 +0x31d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: reflect.Value.call(0xc42037e960, 0xc4202058f0, 0x13, 0x1bda9a0, 0x4, 0xc420f6bd48, 0x3, 0x3, 0x1987f00, 0x19, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/reflect/value.go:447 +0x969 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: reflect.Value.Call(0xc42037e960, 0xc4202058f0, 0x13, 0xc420f6bd48, 0x3, 0x3, 0x16, 0x16, 0x16) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/reflect/value.go:308 +0xa4 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*service).call(0xc4204dd840, 0xc421335720, 0xc420698778, 0x0, 0xc420d1b480, 0xc420bbe380, 0x19758e0, 0xc420d1b900, 0x16, 0x169e020, ...) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:384 +0x14e Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: net/rpc.(*Server).ServeRequest(0xc421335720, 0x1d72340, 0xc4203e4380, 0x0, 0xc420174000) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/net/rpc/server.go:504 +0x235 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).handleNomadConn(0xc420233400, 0x1d71840, 0xc4213725c0, 0x1d7d0e0, 0xc420174000, 0xc421335720) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:236 +0x223 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).handleMultiplex Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/rpc.go:218 +0x2aa Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 177 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4203daac0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 242 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc420192000, 0xc420ad69c0, 0x1d6eb80, 0xc4203daac0, 0xc420ede300, 0xc420ede3c0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420192000, 0xc4203dab80) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 231 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*netPipeline).decodeResponses(0xc4204ddb80) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:667 +0x111 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.newNetPipeline Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/net_transport.go:658 +0x107 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 232 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineDecode(0xc420192000, 0xc420ad66c0, 0x1d6eb80, 0xc4204ddb80, 0xc420e8a480, 0xc420e8a5a0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:448 +0x138 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*Raft).pipelineReplicate.func1() Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/replication.go:385 +0x57 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1(0xc420192000, 0xc4204ddbc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:146 +0x53 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft.(*raftState).goFunc Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/raft/state.go:144 +0x66 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 247 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).serverHealthLoop(0xc420a18640) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:333 +0x14f Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot.(*Autopilot).Start Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/consul/agent/consul/autopilot/autopilot.go:81 +0x136 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 248 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*PlanQueue).Dequeue(0xc420397540, 0x0, 0x0, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_queue.go:155 +0x141 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*Server).planApply(0xc420233400) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/plan_apply.go:59 +0xbf Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*Server).establishLeadership Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/leader.go:192 +0x1ff Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 249 [semacquire]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: sync.runtime_SemacquireMutex(0xc4202218ec, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/runtime/sema.go:71 +0x3d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: sync.(*Mutex).Lock(0xc4202218e8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/sync/mutex.go:134 +0x108 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: sync.(*RWMutex).Lock(0xc4202218e8) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /usr/local/go/src/sync/rwmutex.go:93 +0x2d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*EvalBroker).runDelayedEvalsWatcher(0xc420221860, 0x1d71840, 0xc42204ba80) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:767 +0x294 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*EvalBroker).SetEnabled Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/eval_broker.go:170 +0xf0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 250 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).watchCapacity(0xc42024a630) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:415 +0x12c Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:135 +0x71 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 251 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad.(*BlockedEvals).prune(0xc42024a630) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:619 +0x12d Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad.(*BlockedEvals).SetEnabled Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/blocked_evals.go:136 +0x93 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 252 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*AllocUpdateBatcher).batcher(0xc422e10120) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:67 +0x320 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.NewAllocUpdateBatcher Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/batcher.go:37 +0xd4 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 253 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42204bac0, 0xc420067ce8, 0x20, 0x20, 0x18, 0xc4200182a0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4232081b0, 0x1d71840, 0xc42204bac0, 0xc4200182a0, 0x184987) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420472e40, 0xc422cd7ed0, 0x184987, 0x1d71840, 0xc42204bac0, 0xc420f44260, 0xc422e100f0, 0xc420c11890, 0x24, 0x2bba400) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).getDeploys(0xc420f44200, 0x1d71840, 0xc42204bac0, 0x184987, 0xc42323c080, 0x5, 0x8, 0x184987, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:179 +0x72 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).watchDeployments(0xc420f44200, 0x1d71840, 0xc42204bac0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:151 +0x209 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad/deploymentwatcher.(*Watcher).SetEnabled Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/deploymentwatcher/deployments_watcher.go:124 +0xf2 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 254 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42204bb80, 0xc420f65770, 0x20, 0x20, 0x0, 0x6) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc4230eadb0, 0x1d71840, 0xc42204bb80, 0x0, 0x1849b5) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420472e40, 0xc420f65960, 0x1849b5, 0x1d71840, 0xc42204bb80, 0x552bde, 0xc420f65950, 0xc420f65970, 0xf33ba5, 0xc42050a838) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).getJobAllocs(0xc42050a7e0, 0x1d71840, 0xc42204bb80, 0x1849b5, 0x1, 0x1, 0x0, 0x0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:423 +0xe2 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/drainer.(*drainingJobWatcher).watch(0xc42050a7e0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:145 +0xd26 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad/drainer.NewDrainingJobWatcher Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_jobs.go:88 +0x1ab Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 255 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.watchFew(0x1d71840, 0xc42204bb00, 0xc422cd9a78, 0x20, 0x20, 0x74d2a2, 0x191ee00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch_few.go:16 +0x83a Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb.WatchSet.WatchCtx(0xc423208300, 0x1d71840, 0xc42204bb00, 0xc4232083f0, 0x1833a4) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/hashicorp/go-memdb/watch.go:82 +0x129 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/state.(*StateStore).BlockingQuery(0xc420472e40, 0xc422cd9c68, 0x1833a4, 0x1d71840, 0xc42204bb00, 0x552bde, 0xc420397310, 0x2, 0xc4230fe240, 0x40) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/state/state_store.go:156 +0x175 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).getNodes(0xc42204bbc0, 0x1833a4, 0x3b, 0xc420c95de8, 0x1, 0x1) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:218 +0xd0 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/drainer.(*nodeDrainWatcher).watch(0xc42204bbc0) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:155 +0x7b7 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad/drainer.NewNodeDrainWatcher Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/watch_nodes.go:146 +0xc2 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 256 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/drainer.(*deadlineHeap).watch(0xc42204bc40) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:61 +0x1de Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad/drainer.NewDeadlineHeap Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drain_heap.go:46 +0x127 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: goroutine 257 [select]: Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).run(0xc4205be5a0, 0x1d71840, 0xc42204bb00) Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:226 +0x1e9 Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: created by github.com/hashicorp/nomad/nomad/drainer.(*NodeDrainer).SetEnabled Aug 23 15:29:53 ip-172-24-20-236 nomad.sh[26180]: /opt/gopath/src/github.com/hashicorp/nomad/nomad/drainer/drainer.go:197 +0xe7 Aug 23 15:29:53 ip-172-24-20-236 systemd[1]: nomad.service: Service hold-off time over, scheduling restart. Aug 23 15:29:53 ip-172-24-20-236 systemd[1]: Stopped Nomad agent. Aug 23 15:29:53 ip-172-24-20-236 systemd[1]: Started Nomad agent. Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: WARNING: keyring exists but -encrypt given, using keyring Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: ==> Loaded configuration from /etc/nomad/acl.hcl, /etc/nomad/advertise.hcl, /etc/nomad/consul.hcl, /etc/nomad/server.hcl, /etc/nomad/vault.hcl Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: ==> Starting Nomad agent... Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: ==> Nomad agent configuration: Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: Advertise Addrs: HTTP: 172.24.20.236:4646; RPC: 172.24.20.236:4647; Serf: 172.24.20.236:4648 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: Client: false Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: Log Level: INFO Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: Region: amrv01 (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: Server: true Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: Version: 0.8.4 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: ==> Nomad agent started! Log data will stream in below: Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] raft: Restored from snapshot 10-1583692-1535026670884 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:325d087a-9555-e9e2-eb5d-c186aa7ed41a Address:172.24.20.236:4647} {Suffrage:Voter ID:afff500f-c899-4d26-d003-1a0d1d8bf516 Address:172.24.0.121:4647} {Suffrage:Voter ID:fa7f76f0-0581-92e0-d554-18ae538e5ae8 Address:172.24.10.89:4647}] Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: ip-172-24-20-236.amrv01 172.24.20.236 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.226615 [INFO] nomad: starting 4 scheduling worker(s) for [service batch system _core] Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: Attempting re-join to previously known node: consulnomad-02.aav01: 172.27.0.42:4648 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.226840 [INFO] nomad: adding server ip-172-24-20-236.amrv01 (Addr: 172.24.20.236:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: ip-172-25-5-246.amv01 172.25.5.246 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: ip-172-24-0-121.amrv01 172.24.0.121 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: ip-172-29-4-194.atf01 172.29.4.194 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: ip-172-25-5-210.amv01 172.25.5.210 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [WARN] memberlist: Refuting an alive message Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: consulnomad-01.aaf01 172.26.1.69 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.580593 [INFO] nomad: adding server ip-172-25-5-246.amv01 (Addr: 172.25.5.246:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.580617 [INFO] nomad: adding server ip-172-24-0-121.amrv01 (Addr: 172.24.0.121:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: consulnomad-01.aav01 172.27.0.55 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.580631 [INFO] nomad: adding server ip-172-29-4-194.atf01 (Addr: 172.29.4.194:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.580642 [INFO] nomad: adding server ip-172-25-5-210.amv01 (Addr: 172.25.5.210:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: consulnomad-staging-01.asv01 172.27.0.136 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: ip-172-25-5-80.amv01 172.25.5.80 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: vol-cl-control-03.global 172.16.9.35 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: vol-cl-control-01.global 172.16.9.87 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: ip-172-29-20-70.atf01 172.29.20.70 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.580895 [INFO] nomad: adding server consulnomad-01.aaf01 (Addr: 172.26.1.69:4647) (DC: aaf01) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.580912 [INFO] nomad: adding server consulnomad-01.aav01 (Addr: 172.27.0.55:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.580924 [INFO] nomad: adding server consulnomad-staging-01.asv01 (Addr: 172.27.0.136:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.580935 [INFO] nomad: adding server ip-172-25-5-80.amv01 (Addr: 172.25.5.80:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: vol-cl-control-02.global 172.16.9.89 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.580946 [INFO] nomad: adding server vol-cl-control-03.global (Addr: 172.16.9.35:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.580957 [INFO] nomad: adding server vol-cl-control-01.global (Addr: 172.16.9.87:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: consulnomad-02.aaf01 172.26.1.191 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: consulnomad-03.aaf01 172.26.1.78 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: consulnomad-02.atf01 172.29.0.72 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.581196 [INFO] nomad: adding server ip-172-29-20-70.atf01 (Addr: 172.29.20.70:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.581211 [INFO] nomad: adding server vol-cl-control-02.global (Addr: 172.16.9.89:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.581223 [INFO] nomad: adding server consulnomad-02.aaf01 (Addr: 172.26.1.191:4647) (DC: aaf01) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.581234 [INFO] nomad: adding server consulnomad-03.aaf01 (Addr: 172.26.1.78:4647) (DC: aaf01) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.581245 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: consulnomad-staging-03.asv01 172.27.0.64 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.581281 [INFO] nomad: adding server consulnomad-02.atf01 (Addr: 172.29.0.72:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.581296 [INFO] nomad: adding server consulnomad-staging-03.asv01 (Addr: 172.27.0.64:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: consulnomad-02.aav01 172.27.0.42 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: consulnomad-staging-02.asv01 172.27.0.200 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: EventMemberJoin: ip-172-27-0-204.aav01 172.27.0.204 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54 [INFO] serf: Re-joined to previously known node: consulnomad-02.aav01: 172.27.0.42:4648 Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.581606 [INFO] nomad: adding server consulnomad-02.aav01 (Addr: 172.27.0.42:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.581623 [INFO] nomad: adding server consulnomad-staging-02.asv01 (Addr: 172.27.0.200:4647) (DC: test) Aug 23 15:29:54 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:54.581635 [INFO] nomad: adding server ip-172-27-0-204.aav01 (Addr: 172.27.0.204:4647) (DC: test) Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] state_store: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "9b266675-75b9-c60c-79a6-66a0c19c9feb" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] state_store: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:56 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:56 [ERR] nomad.fsm: ApplyPlan failed: unable to find eval id "c646ec26-e575-b5a7-2266-bf470fe6810d" Aug 23 15:29:57 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:57 [INFO] serf: EventMemberUpdate: ip-172-24-0-121.amrv01 Aug 23 15:29:58 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:58 [WARN] raft: Heartbeat timeout from "172.24.0.121:4647" reached, starting election Aug 23 15:29:58 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:58 [INFO] raft: Node at 172.24.20.236:4647 [Candidate] entering Candidate state in term 83 Aug 23 15:29:58 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:58 [INFO] raft: Node at 172.24.20.236:4647 [Follower] entering Follower state (Leader: "") Aug 23 15:29:58 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:29:58 [WARN] raft: Failed to get previous log: 1598492 log not found (last: 1598490) Aug 23 15:30:00 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:30:00.471484 [INFO] server.nomad: successfully contacted 20 Nomad Servers Aug 23 15:32:48 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:32:48 [INFO] raft: Starting snapshot up to 1599916 Aug 23 15:32:48 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:32:48 [INFO] snapshot: Creating new snapshot at /var/lib/nomad/server/raft/snapshots/84-1599916-1535027568185.tmp Aug 23 15:32:48 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:32:48 [INFO] snapshot: reaping snapshot /var/lib/nomad/server/raft/snapshots/10-1553610-1535026056664 Aug 23 15:32:48 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:32:48 [INFO] raft: Compacting logs from 1573463 to 1589676 Aug 23 15:32:48 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:32:48 [INFO] raft: Snapshot to 1599916 complete Aug 23 15:35:31 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:35:31 [INFO] serf: EventMemberFailed: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:35:31 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:35:31.857751 [INFO] nomad: removing server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:35:34 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:35:34 [INFO] memberlist: Suspect ip-172-24-10-89.amrv01 has failed, no acks received Aug 23 15:36:45 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:45 [INFO] serf: EventMemberJoin: ip-172-24-10-89.amrv01 172.24.10.89 Aug 23 15:36:45 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:45.293028 [INFO] nomad: adding server ip-172-24-10-89.amrv01 (Addr: 172.24.10.89:4647) (DC: test) Aug 23 15:36:45 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:45 [WARN] raft: Rejecting vote request from 172.24.10.89:4647 since we have a leader: 172.24.0.121:4647 Aug 23 15:36:46 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:46 [WARN] raft: Rejecting vote request from 172.24.10.89:4647 since we have a leader: 172.24.0.121:4647 Aug 23 15:36:48 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:48 [WARN] raft: Rejecting vote request from 172.24.10.89:4647 since we have a leader: 172.24.0.121:4647 Aug 23 15:36:50 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:50 [WARN] raft: Rejecting vote request from 172.24.10.89:4647 since we have a leader: 172.24.0.121:4647 Aug 23 15:36:50 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:50.790594 [ERR] worker: failed to dequeue evaluation: rpc error: eval broker disabled Aug 23 15:36:50 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:50.790636 [ERR] worker: failed to dequeue evaluation: rpc error: eval broker disabled Aug 23 15:36:50 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:50.790647 [ERR] worker: failed to dequeue evaluation: rpc error: eval broker disabled Aug 23 15:36:50 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:50.790683 [ERR] worker: failed to dequeue evaluation: rpc error: eval broker disabled Aug 23 15:36:51 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:51 [WARN] raft: Rejecting vote request from 172.24.10.89:4647 since we have a leader: 172.24.0.121:4647 Aug 23 15:36:52 ip-172-24-20-236 nomad.sh[26258]: 2018/08/23 15:36:52.167848 [ERR] worker: failed to dequeue evaluation: rpc error: eval broker disabled