Skip to content

v1.8.2

Latest
Latest
Compare
Choose a tag to compare
@Duyi-Wang Duyi-Wang released this 10 Oct 08:17
· 2 commits to main since this release
v1.8.2
b43edc8
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

v1.8.2

Performance

  • Enable flash attention by default for W8A8 dtype to accelerate the performance of the 1st token.

Benchmark

  • When the number of ranks is 1, run in single mode to avoid the dependency on mpirun.
  • Support SNC-3 platform.
Assets 2
Loading