Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the smaller the value of beta is ,the more difficult the item is ? #20

Open
rudy-letote opened this issue May 25, 2020 · 1 comment
Open

Comments

@rudy-letote
Copy link

rudy-letote commented May 25, 2020

hi ,there is a problem which confuse me a few of days .i run code as follows:
res = irt(data)
i check many items in the data. when the beta is smaller such as -1.9,-2 , but the liked rate of this item is very low which i think it should be more large.when the beta is bigger such as 1.9 , 2 ,but the liked rate of this item is very high which i think it should be smaller. could anyone explain sth to me when the beta is large ,the liked rate should be large or smaller. note: the liked rate equals the number of "1" divide all numbers of items .
the format of the data is as follows :

customer_id product_id liked
1086992 14 1
1112695 14 0
1112695 17 1
803222 18 1
1110363 19 0
273880 19 1
445253 19 0
@sadrasabouri
Copy link

I think there is a mistake in the library (maybe the real b is -b).
I ran this toy example to check. Clearly, the first question was the hardest since everyone correctly evaluated it, and the second one was the hardest since they all missed it.

from pyirt import irt
from pprint import pprint

item_param, user_param = irt([
    (1, 1, 0), (1, 2, 1), (1, 3, 1),
    (2, 1, 0), (2, 2, 1), (2, 3, 0),
    (3, 1, 0), (2, 2, 1), (2, 3, 1),
])

pprint(item_param)
# {1: {'alpha': 0.25, 'beta': -2.0, 'c': 0.0},
# 2: {'alpha': 0.5720948359800555, 'beta': 2.0, 'c': 0.0},
# 3: {'alpha': 0.25, 'beta': 0.634102890032698, 'c': 0.0}}
pprint(user_param)
# {1: 0.13484234439295528, 2: 0.07287218993120267, 3: 0.019347314772223796}

Based on the original paper in equation (4), the probability of successfully answering questions is as follows:

$$ P( y_{ij}=1 | a_i, b_i, c_i=0, \theta_j) = \frac{1}{1 + e^{-a_i(\theta_j - b_i)}} $$

Since $a_i>0$ given a fixed value for subject ability, $\theta_j$, higher values of $b_i$ should result in a lower probability of people getting the answer right. But that's not the case here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants