How to handle maximum document size?

[codeliner]

Question extracted from prooph/proophessor#20

About maximum document size: We can perhabs have 2 adapters?
The first one has the 16MB per document limitation, but will work slightly faster. The second one stores a reference to a mongo grid fs entry and has no limitations at all.

[codeliner]

prooph/event-store provides different stream strategies. They operate on aggregate level and invoke the event store differently when an aggregate is going to be added or updated.

All other PHP event store implementations (I'm aware of) only support a single event stream and fetch all events for an aggregate from the ES by using the UUID of the aggregate as filter.

1. The same is supported by prooph/event-store with the SingleStreamStrategy. For mongoDB this would mean that we have a event stream documentset and each event is a document. In this case we won't run into the limit because a single event with more than 16 MB doesn't make sense, right?
   Disadvantage is, that transaction handling is a bit tricky.
2. When using the AggregateStreamStrategy (and the AggregateTypeStreamStrategy would nearly be the same) the documentset would be something like users and each aggregate would be a document with its own event stream:

{
_id : //aggregate root id
event_steam : [
{
event_id : 
event_name :
//....
},
//... more events here
]
}

👍 : Transaction handling is simple because updating the event stream of the aggregate can be done within one atomic mongoDB operation, right? When looking at DDD an aggregate should be consistent but between multiple aggregates eventual consistency is preferred. So the atomic update operation would be enough.
👎 16 MB limit

I think we should provide both options so people can choose the one that best fits their needs (by using the adapter together with the appropriate StreamStrategy). But I'm not sure if this can be implemented in one adapter or if we really need two to keep them simple and focused.
What do you think @prolic?

[prolic]

I'll try to play a little with some implementation tomorrow and let you know what I got ;-)

[codeliner]

👍 some working code would be great

[codeliner]

Ping @prolic: Did you find some time to play with ideas or should I start working on it (if I find some time in the next weeks ;))

[prolic]

Sorry, first I got sick and then I needed to rearrange some timing
priorities. This will definetly come the next 2-3 weeks. Very sorry for the
delay, but there is also much more to expect, like rabbit-mq-extensions :)

Best

Sascha

Sascha-Oliver Prolic

2015-07-03 19:50 GMT+02:00 Alexander Miertsch notifications@github.com:

Ping @prolic https://github.com/prolic: Did you find some time to play
with ideas or should I start working on it (if I find some time in the next
weeks ;))

—
Reply to this email directly or view it on GitHub
#3 (comment)
.

[codeliner]

No problem. And very good news. Thanks for the update.
I definetly need help with mongo and rabbit. I'm not an expert in both but these are very important features. If I can help you with an issue in your repos let me know.
Thanks again.

[codeliner]

@prolic Can we close the issue as we are using one document per event now?
I think an event with more than 16 MB of data is an edge-case and can be handled in another way if required.

[prolic]

I am not sure. What about file uploads?
Am 09.08.2015 12:24 schrieb "Alexander Miertsch" notifications@github.com:

@prolic https://github.com/prolic Can we close the issue as we are
using one document per event?
I think an event with more than 16 MB of data is an edge-case and can be
handled in another way if required.

—
Reply to this email directly or view it on GitHub
#3 (comment)
.

[codeliner]

I consider file uploads as one of the edge-cases 😄 It is not recommended to put uploaded files into events but a reference to the file.

[prolic]

Ok, so we can close this, I think ;)