Has anyone tried Dolly-like models?

[kha84]

I just watched the latest video of my favorite youtuber - https://www.youtube.com/watch?v=AWAo4iyNWGc&t=14s and was wondering, if someone has already quantized & converted one of these to be compatible with llama.cpp?
The beauty of Dolly-like models is that they're based on open source gpt-j-6B from EleutherAI, so noone will be hunting us for using them without an ask.

[pikalover6]

ggml already supports gpt-j, you should just be able to convert and quantize them.

[gorborukov]

I did not succeed with the convert-h5-to-ggml.py script and the model Dolly_GPT-J-6b. Script thinks for a while and crashes with the status "Killed".

[kha84]

I did not succeed with the convert-h5-to-ggml.py script and the model Dolly_GPT-J-6b. Script thinks for a while and crashes with the status "Killed".

Prob out-of-memory killer played his part? Check journalctl / syslog and try to monitor system resources while running it