Skip to content

Commit

Permalink
Revert "remove change breaking sharktank benchmarks"
Browse files Browse the repository at this point in the history
This reverts commit c617cfb.
  • Loading branch information
renxida committed Dec 3, 2024
1 parent c617cfb commit 0f00b1f
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion sharktank/sharktank/examples/export_paged_llm_v1.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,9 @@ def generate_params_json(hp, prefill_bs: list[int], decode_bs: list[int]):
"prefill_batch_sizes": prefill_bs,
"decode_batch_sizes": decode_bs,
"transformer_block_count": hp.block_count,
"block_seq_stride": llama_config.block_seq_stride,
"paged_kv_cache": {
"attention_head_count_kv": hp.attention_head_count_kv,
"block_seq_stride": llama_config.block_seq_stride,
"device_block_count": 256, # so that this makes its way into the config file & can be edited.
},
}
Expand Down

0 comments on commit 0f00b1f

Please sign in to comment.