Storm 1.0.0 Support #128
Comments
|
Spent a little time, and I got it to compile with 1.0.0, fortunately all issues with the executable code are changes to packages names Unfortunately, the tests are a little more broken as a few constructors are broken. Open to suggestions. |
|
@DarinJ : Can you please push a branch with the refactor you did so far? Tried some sed'ing like you suggested, but have a bunch of issues still. |
|
@erikdw take a look at https://github.com/DarinJ/storm/tree/1.0.0-WIP, I'm not at a box with git right now but I can push to a like branch later. |
|
Sigh, |
|
Yeah a few c style ifdefs would be awesome right now :(. I'm also mostly still 0.9.6 now and only testing 0.10.x. I'll look into it some more later but thought I'd start the conversation. |
|
Pushed my changes: @DarinJ : so I got the tests to work for
Other main changes:
|
Here's the MesosNimbus and ui running on storm-1.0.0:Haven't yet rebuilt a topology and tried launching it
|
Here's proof of the MesosSupervisor and worker running on storm-1.0.0:
|
|
@erikdw you've been busy! Thanks, I'll check it out tomorrow. On Tue, Apr 12, 2016 at 10:16 PM, Erik Weathers notifications@github.com
|
|
@erikdw 👍 very nice. Great Job :) For the support , we can decide to make storm-mesos 0.2.0 compatible only with storm 1.0.0 and then remove all the code from 0.9.x and 0.10.x. That would clean up a lot of things and be leaner. What about HA from storm 1.0.0, have you done anything there ? I'm very excited for storm 1.0.0 :) Thanks |
|
@salimane : Thanks! Regarding rev'ing the project up to 0.2.0 or 1.0 or something and dropping earlier storm version support: unfortunately, I still have some things I wanna update in storm-mesos running on storm-0.9.x before declaring it "done". Specifically, we have some changes for the slot scheduling that greatly improve its handling of corner cases. Sadly, we cannot focus on getting those in yet, because we're in the midst of a large revamp to get off of a hacky fork onto this OSS project. As soon as we get that in a solid state we'll come back to fixing the scheduler. Also, given what I've revealed before (having dozens of teams with their own topology code that all needs to be updated for storm-1.0), just dropping storm-0.9.x and moving to storm-1.0 is going to require a ton of coordination, so it's not something we can do at the drop of a hat. @salimane & @DarinJ : I was thinking about how to handle this huge deviation with storm-1.0 renaming the package paths ( The alternative would appear to be an extension of @salimane 's proposal: create separate release trains on separate branches, with 0.1.x being left as the one that storm-0.9.x & storm-0.10.x are built for, and master becoming "latest" for now. The implication is that we'll have to selectively backport changes to the 0.1.x branch. Though that sucks, it might be easier and less dirty long-term than the horrible hackishness I just proposed in the previous paragraph. |
|
@salimane : also, regarding this question:
Nope, haven't done anything, except that I noticed the interesting thing in the screenshot of the Storm UI: 2 Nimbuses sitting there! |
|
@erikdw, yep as you said in the second paragraph, we can still do minor update to storm-mesos 0.1.x and have 0.1.1, 0.1.2... etc. Then 0.2.0 would be only for storm 1.0.0 + They are using the same way with apache storm. https://github.com/apache/storm/tree/1.x-branch is for the |
|
@salimane : Oh, and the fact that the travis deploy integration is based on the git tag should allow us to trivially continue making releases from each branch (0.1.x and master) right? |
|
yep exactly, you are right :) |
|
@erikdw We've been playing with our mod's a bit which ended up working pretty well. While we're not in a hurry to jump to storm 1.0.0, we'll probably skip 0.10.X and jump to 1.0.0. I'll explore the option of shim v branch to see if it's worth while, though I'm not optimistic with the number of method signatures that changed in the unit tests. There are still a number of additional things that should be addressed wrt 1.0.0 such as does HA nimbus just work and should we launch a pace maker and if so how should it be handled wrt service discovery. I'll probably start work in a couple weeks as I'm currently on a different project unless someone beats me to it. |
|
@DarinJ : sounds good, we're also busy for the moment. And I'm also excited to figure out how to get HA Nimbus working directly on Mesos. Regarding "pacemaker" (the hearbeat handling server) I remember reading something, but I'm not familiar yet. I assume we should set it up too... but don't "know" for sure. And yeah, I have little faith that the effort for shimming everything will be worth it, and also am worried about super uglifying the code / maven settings to accomplish it. |
|
any update on this ? :( |
|
@salimane not really I planned to do some work this weekend but decided to go strawberry picking instead. Hoping to get to it soon. Definitely going with a separate branch though, I determined shims were to much work. |
|
@salimane & @DarinJ & @dsKarthick : yeah, branches seem the way to go. I have work time allocated for this during the next 2 weeks, so I'll work on it. Here's the proposal:
|
|
FYI, I'm holding off working on the branch-split until @dsKarthick 's big change in PR #146 lands, since it's a bunch of code. We'll then let it bake for a week or so in our staging environment, after which I will proceed to perform this split and we can start iterating on storm-1.0+ support in the master branch. |
|
@erikdw I built Storm 1.0 and mesos 0.28.2 , but failure . |
|
@erikdw when I built Storm 1.0 and mesos 0.28.2,meet a problem. |
|
@zhanghangchina : (I deleted the redundant message you posted in this issue) Can you clarify where you read this message? I am not sure you you mean by "It says". Let's talk about this separately, I don't wanna spoil this (#128) with this tangent, which is likely to be something related to your setup, 'cause I have used this stuff just fine myself. So like I said in PR #167, please open an issue. Or for quicker response you can log into the gitter.im room we have: https://gitter.im/mesos/storm |
|
@awesome ! |
|
Complete. See releases here: |
Storm 1.0.0 just landed, I'm exploring what it'll take to get it running on Mesos. Not sure the level of difficulty in this, but thought I'd open the thread.
The text was updated successfully, but these errors were encountered: