Skip to content
This repository has been archived by the owner on Jan 20, 2022. It is now read-only.

Reducer estimation does not work with summingbird #713

Open
johnynek opened this issue Feb 9, 2017 · 0 comments
Open

Reducer estimation does not work with summingbird #713

johnynek opened this issue Feb 9, 2017 · 0 comments

Comments

@johnynek
Copy link
Collaborator

johnynek commented Feb 9, 2017

In scalding, Reducer estimation runs in this code:
https://github.com/twitter/scalding/blob/72ef68b465b17fcb56a29fc13ccb8b1013430b49/scalding-core/src/main/scala/com/twitter/scalding/ExecutionContext.scala#L91

but that code is not called in summingbird:

def toFlow(config: Config, timeSpan: Interval[Timestamp], mode: Mode, pf: PipeFactory[_]): Try[(Interval[Timestamp], Option[Flow[_]])] = {

so, the options are ignored.

if we solved #710 this would probably just go away.

# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant