Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I would like to define a new scoring in GridSearchCV as it is said here http://scikit-learn.org/stable/modules/model_evaluation.html#implementing-your-own-scoring-object . This is my code:

from sklearn.model_selection import GridSearchCV
def pe_score(estimator,x,y):
    clf=estimator
    clf.fit(x,y)
    z=clf.predict(x)
    pe=prob_error(z, y)
    return pe
pe_error=pe_score(SVC(),xTrain,yTrain)
grid = GridSearchCV(SVC(), param_grid={'kernel':('linear', 'rbf'), 'C':[1, 10, 100,1000,10000]}, scoring=pe_error)
where prob_error(z,y) is the function that computes the error which I would like to minimize, being z the prediction of the training set and y the true values of the training set. However, I got the following error:
---> 18 clf.fit(xTrain, yTrain)
TypeError: 'numpy.float64' object is not callable
I don't know if the format of pe_error it is well defined. How can I solve it? Thank you.
                Your code contains scoring=ftwo_scorer which makes me think you're not showing all of the relevant sections of source code. What is this scorer function?
– ely
                Nov 20, 2017 at 13:25
                In this code block, you're also not importing SVC, which means there is still more code that you're not revealing. The particular error you show suggests there is a whole different way in which you are invoking clf.fit. The best thing would be to just put all of the code here without trying to extract the part that you currently suspect is in error.
– ely
                Nov 20, 2017 at 14:31
Score functions should have the format score_func(y, y_pred, **kwargs)
You can then use the make_scorer function to take your scoring function and get it to work with GridSearchCV.
So, in this case it would be:
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import make_scorer
clf = estimator
clf.fit(x,y)
z = clf.predict(x)
def pe_score(y, y_pred):
    pe = prob_error(y_pred, y)
    return pe
pe_error = make_scorer(pe_score)
grid = GridSearchCV(SVC(), param_grid={'kernel':('linear', 'rbf'), 'C':[1, 10, 100,1000,10000]}, scoring= pe_error)
(I'm assuming you have prob_error implemented or imported somewhere else in your code)
Documentation: http://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html
                Ok, one question, as I have to minimize the error should I write?:     pe_error = make_scorer(pe_score, greater_is_better=False)
– xelact
                Nov 21, 2017 at 11:06
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.