Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Failed to restore state when open checkpoint option #256

Open
daugraph opened this issue Oct 7, 2023 · 0 comments · May be fixed by #257
Open

Failed to restore state when open checkpoint option #256

daugraph opened this issue Oct 7, 2023 · 0 comments · May be fixed by #257

Comments

@daugraph
Copy link

daugraph commented Oct 7, 2023

Thanks for you work, it seems SlidingWindow doesn't support restore state from CheckPoint

2023-10-07 16:22:31
java.io.IOException: Exception while applying AggregateFunction in aggregating state
	at org.apache.flink.runtime.state.heap.HeapAggregatingState.add(HeapAggregatingState.java:107)
	at org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.processElement(WindowOperator.java:404)
	at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)
	at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)
	at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:519)
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:804)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:753)
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:963)
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:942)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:756)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:578)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Serializer does not support named field positions.
	at org.apache.flink.api.java.typeutils.runtime.RowSerializer.copyNameBased(RowSerializer.java:173)
	at org.apache.flink.api.java.typeutils.runtime.RowSerializer.copy(RowSerializer.java:144)
	at org.apache.flink.api.java.typeutils.runtime.RowSerializer.copy(RowSerializer.java:72)
	at org.apache.flink.runtime.state.heap.CopyOnWriteStateMap.transform(CopyOnWriteStateMap.java:375)
	at org.apache.flink.runtime.state.heap.StateTable.transform(StateTable.java:211)
	at org.apache.flink.runtime.state.heap.HeapAggregatingState.add(HeapAggregatingState.java:105)
	... 14 more

I guess it is caused by we created an name-based row in aggregation function

SlidingWindowUtils.java#L227

but flink row serializer doesn't support name-based row.

RowSerializer.java#L170

daugraph added a commit to daugraph/feathub that referenced this issue Oct 8, 2023
daugraph added a commit to daugraph/feathub that referenced this issue Oct 8, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant