salesforce: autofetch is broken

[josephjclark]

The autofetch feature on the salesforce query sort of works but also not really.

This PR #478 introduced a few regressions:

• Referneces is nested, so you have to do [0][0]
• The adaptor pushes responses into references, not recerds, So the data isn't really even accurate

Basically autofetch needs to push the RECORDS into a single flat results array

Not to mention that the code is a bit quirky, to say the least.

I'd also ask whether callback should be triggered per batch, or at the end result. I think that when you're talking about a batch processing API it make sense to chunkup the callback or else you're likely to hit a problem downstream. Maybe this is an option. This is something that comes up on many APIs so I'd want to think carefully about what we do here.

[josephjclark]

And there are no unit tests on any of this functionality

[mtuchi]

EOD update

I have started working on this but i need @josephjclark input. I think we shouldn't combine records into a single array because the other metadata will be lost (See screenshot attached below 👇🏽 ).
If we can rule out that the other metadata Eg: done, nextRecordsUrl, totalSize are not important when we use autoFetch then yeah we can merge the records into a single array

[josephjclark]

The other metadata is not useful when using autofetch. We'll use that stuff internally but there's no point feeding it back to the user.

[aleksa-krolls]

Agreed with Joe's comment.