Skip to content

November Binary Update #962

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 7 commits into from
Nov 7, 2024
Merged

Conversation

martindevans
Copy link
Member

@martindevans martindevans commented Oct 26, 2024

Updated to 958367bf530d943a902afa1ce1c342476098576b, built with https://github.com/SciSharp/LLamaSharp/actions/runs/11509292954

This is a much smaller update that the last one. Only notable changes are:

  • Some changes in the fill-in-middle (FIM) token functions (no change to C# API)
  • New FIM tokens exposed, not sure what these are for so no useful comments, if anyone knows a tip would be appreciated!
  • Removed softmax sampler. It's deprecated upstream, so it's best to remove it quickly on our end since that's a new API no one will be depending on yet.
  • Removed tail free sampler. Also deprecated upstream (llama : remove Tail-Free sampling ggml-org/llama.cpp#10071)

Testing:

  • Windows CPU
  • Windows CUDA
  • Windows Vulkan
  • Linux CPU
  • Linux CUDA
  • Linux Vulkan
  • MacOS CPU
  • MacOS Metal

@m0nsky
Copy link
Contributor

m0nsky commented Oct 26, 2024

Unit tests passed on Windows CUDA.

Test application is running fine on:

  • Windows CPU
  • Windows CUDA
  • Windows Vulkan
  • Linux CPU
  • Linux CUDA
  • Linux Vulkan

@martindevans martindevans requested a review from SignalRT November 1, 2024 21:20
@martindevans martindevans merged commit b321839 into SciSharp:master Nov 7, 2024
6 checks passed
@martindevans martindevans deleted the nov_binaries branch November 7, 2024 02:37
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants