Skip to content

stormprocessor/storm-scribe

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 

Repository files navigation

This is an implementation of a spout that receives data from Scribe. It is an example of a "push spout", in that data is pushed to it rather than the spout pulling data from somewhere else.

This spout requires a fork of Scribe that allows for Zookeeper-based node discovery: https://github.com/traviscrawford/scribe

See http://groups.google.com/group/scribe-server/msg/c6d04c3b0ee25ddc for more details on how that works.

This is an unreliable spout that doesn't guarantee data processing. With some more work, it can be extended to be reliable – see the note in ScribeReceiver.java.

This spout *mostly* respects TOPOLOGY_MAX_SPOUT_PENDING. If the spout is deactivated (like as part of the topology shutdown), it will flush its internal queue of messages to the output collector (which may be more than TOPOLOGY_MAX_SPOUT_PENDING). This ensures that every message pushed to it is at least attempted to be processed. As explained above, with more work actual reliability could be added to the spout.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages