Unofficial of Chain-of-Thought Decoding @DeepMind
Great Work from Xuezhi Wang and Denny Zhou :-> Paper link: https://arxiv.org/abs/2402.10200
Any HF model can be plugged-in and play
from decode import *
query="A coin is heads up. Fletcher flips the coin. Conception flips the coin. Is the coin still heads up?"
template = """[INST]{question}[/INST]"""
k_response = get_k_path_prob(model, tokenizer, template.format(question=query), k=5)
k_response is a list of word and its probability like this:
[('There', 0.005),
('is', 0.025),
('no', 0.965),
('certainty', 0.25),
('about', 0.015),
('the', 0.002),
('coin', 0.0),
('still', 0.0),
('being', 0.0),
('heads', 0.011),
('up', 0.001),
('after', 0.0),
('both', 0.092),
('flips', 0.072)]
On of the Issue with the original paper, is that it's difficult to locate the answer's position in the response. With a follow up question of "So the answer is", it is easier to parse the answer
from decode import *
query="A coin is heads up. Fletcher flips the coin. Conception flips the coin. Is the coin still heads up?"
template = """[INST]{question}[/INST]"""
k_follow_up_response = get_k_path_prob_follow_up(model, tokenizer, template.format(question=query), k=5)
![Screenshot 2024-02-23 at 08 37 31](https://private-user-images.githubusercontent.com/66006349/307194960-d30dcb38-e207-429e-a8d6-2f13c62a3503.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxMDI2NDIsIm5iZiI6MTcyNjEwMjM0MiwicGF0aCI6Ii82NjAwNjM0OS8zMDcxOTQ5NjAtZDMwZGNiMzgtZTIwNy00MjllLWE4ZDYtMmYxM2M2MmEzNTAzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MTIlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTEyVDAwNTIyMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWI0YjdjNGFmOWYyMjBmYTAyMTEzZDQxNWI2MmRkZDRkYzJjNzZlNmI3MjIyMjMyMDQ2OGY2YWI0ZDM1YjM4MjAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.1dULaLhSyT049yKa9zSzUB6AWJsZAa9oB1dIW4r_8oI)