# SVM (support vector machine) with libsvm

I am learning svm lately and tried libsvm. It’s a good package.

Linear kernel example (support vectors are in circles):

3-class example

Basic procedure to use libsvm:

1. Preprocess your data. This including normalization (make all values between 0 and 1) and transform non-numeric values to numeric. You can use the following code to normalize (from libsvm webpage):
`(data - repmat(min(data,[],1),size(data,1),1))*spdiags(1./(max(data,[],1)-min(data,[],1))',0,size(data,2),size(data,2))`
2. Find optimal parameter values. For linear kernel, you have 1 parameter C (penalize parameter). For commonly used radial kernel, you have two parameters (C and gamma). Different parameter values will yield different accuracy rate. To avoid over fitting, you use n-fold cross validation. For example, a 5-fold cross validation is to use 4/5 of the data to train the svm model and the rest 1/5 to test. The option -c, -g, and -v controls parameter C, gamma and n-fold cross validation. A piece of code from libsvm website is:
```bestcv = 0;
for log2c = -1:3,
for log2g = -4:1,
cmd = ['-v 5 -c ', num2str(2^log2c), ' -g ', num2str(2^log2g)];
cv = svmtrain(heart_scale_label, heart_scale_inst, cmd);
if (cv >= bestcv),
bestcv = cv; bestc = 2^log2c; bestg = 2^log2g;
end
fprintf('%g %g %g (best c=%g, g=%g, rate=%g)\n', log2c, log2g, cv, bestc, bestg, bestcv);
end
end```
3. You may have to run the above code several times with different range of parameter values to find the optimal values. For example, you might want to start from a bigger range with coarse resolution; then fine tune to smaller regions with higher resolution.
4. After finding the optimal parameter values, use all data to train your model with your optimal parameter values.
```cmd = ['-t 2 -c ', num2str(bestc), ' -g ', num2str(bestg)];
model = svmtrain(l, d, cmd);```
5. If you have new data, you may use this model to classify the new data.
`[predicted_label, accuracy, decision_values] = svmpredict(zeros(size(dd,1),1), dd, model);`

Commonly used options

• -v n: n-fold cross validation
• -t 0: linear kernel
• -t 2: radial basis (default)
• -s 0: SVC type = C-SVC
• -C: C parameter value, default 1
• -g: gamma parameter value

libsvm performance

I tested on different data size and record the time spent (in second).

Computer: Processor: 2×2.66G, memory: 12G, OS: Windows XP installed in VMWare in Mac OS 10.5

data size    # features    svmtrain    svmpredict
100    2    0.00    0.00
100    6    0.00    0.00
100    10    0.00    0.00
100    20    0.00    0.00
100    50    0.01    0.00
100    100    0.02    0.01
500    2    0.02    0.01
500    6    0.03    0.02
500    10    0.05    0.03
500    20    0.08    0.03
500    50    0.46    0.07
500    100    0.56    0.12
1000    2    0.07    0.04
1000    6    0.10    0.06
1000    10    0.15    0.10
1000    20    0.36    0.14
1000    50    1.09    0.30
1000    100    3.07    0.50

It’s fairly fast.

Resources:

MatLab code to generate the plots above:cuixu_test_svm1

SVM basics: http://en.wikipedia.org/wiki/Support_vector_machine

The meaning of libsvm output is at: http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#f804

## Calculate phase difference between two general signals (e.g. HbO…

In a recent fNIRS journal club (vedio recorded here), Dr. Tong talked about their work on the phase difference between oxy and deoxy Hb, and its relationship with participants’ age. This article is a demo of how to use Hilbert transform to calc

## Functional Near Infrared Spectroscopy (fNIRS): List of manufactures

Contents Company Products Artinis Brite, OxyMon, OctaMon, PortaMon, PortaLite BIOPAC fNIR 100, fNIR2000M Series CORTIVISION PHOTONCAP Huichuang Medical NirScan-900X, NirSmartⅡ- Pro GOWERLABS NTS fNIRS system, LUMO HAMAMATSU NIRO-200NX HITACHI ETG-410

## 58 Replies to “SVM (support vector machine) with libsvm”

1. Florian says:

Hi Xu Cui,

I am working also with LibSVM and I am very interested in your work 🙂
I have some questions and therefore it would be great if we can discuss it via e-mail.

Best regards
Florian

2. erol says:

Hi Xu Cui,
I am working with libsvm newely and I want to write a simple c++ program for one againts rest svm.I will be happy if you help me.
thanks,
erol

3. Enigma says:

Is there a way to see the voting process for the one vs all implementation in LIBSVM?

4. PG says:

Hi Xu Cui,

I have estimated SVM parameter thrugh some of the papers using model selection criteria, now I want to test my prediction model using cross-validation as it gives generalized error. will In svmtrain option if I give -v 5 my code shall work, kindly let me know. I’m using MATLAB version n quite poor in C++, so help me out

5. Xu Cui says:

@PG
Yes, -v 5 should work.

6. David says:

Hi Xu Cui,

Your SVM plotting code for “Linear – 3 classes” only seems to plot a 2-dimensional vector. If I had a matrix of 3 or 4-dimensional vectors, how would the code be modified so that it can represent a +2-dimensional vector accurately in the output plot?

7. Xu Cui says:

Maybe you could project your high dimensional data to 2D?

8. raied says:

Dear Xu Cui,
I found the Matlab code for LIBSVM is very helpful. However, I would like to comment on one point you mentioned about the testing data. In your code, in the linear part, you claimed that half of the data is for training and the other half for testing. I can’t see that reflected in the published code. As a matter of fact the testing data you use is only coming from the use of “meshgrid” and it does not resemble half of the data.

If you have another version of the code then that will be very good.

Thanks

Raied

9. serra says:

Dear Xu Cui,
I have a very interesting and weird question about SVM. When I use a different set (9 dimensions) of image and a reference image that doesn’t belong to it, the result of poly -SVM is giving %51 percent of accuracy.
How does this possible? Is taht mean I am doing smthg wrong or Lib-SVM is doing wrong.I can send the code if u want.
Thanks

10. Xu Cui says:

@serra
I wonder why is it not possible to get 51%. Yes, code would be helpful.

BTW, what is poly-SVM?

11. Xu Cui says:

@raied
Thank you for letting me know, raied.

12. serra says:

thanks for your effort, but I get an error saying;
”??? Error using ==> svmtrain at 172
Group must be a vector.”
in the cv = svmtrain(l, d, cmd); line,
and when i want to drow SVs it draws only 2 circles in irrelevant places? what is wrong i didn’t understand?
I need help, tahnk you in advace
Regards,

13. sirine says:

dear Xu Cui
i have to use LibSvm for a multiclass classification so actually i’m using libsvm 3.0 for matlab. i want to know if the code above ( which generate the plots) support a multiclass classification?? if not how can i change it to support a multiclass classification
Thank you
yours

14. amir says:

could you give me some example for libsvm matlab for classification and espicially regression?
i really need matlab mfile of regression using libsvm

thanks

15. Hello there, You’ve done a fantastic job. I will definitely digg it and personally suggest to my friends. I am confident they’ll be benefited from this website.

16. Bui says:

Hi Xu Cui

I used your SVM code (RBF) for my data. I have two classes, coded 0.1 and 0.9
I scaled the data in a rank [0.1 0.9], 10 variables and 1368 observations

Accuracy of The train data is good. But the testing data (3080 observation) is 0% or very low

http://ifile.it/d0svzmt/Data.rar

I need you help to check.

Thanks

17. Aakash says:

Hey, nice article. Was a great help. I want to use multi-class SVM (one-vs-rest approach). I have 6 labels and each label has 1000 feature. Can you please guide me in the right direction.
Thanks

18. Demir says:

Dear Xu Cui,

While I was searching for probability estimates for SVR as an output of
LIBSVM, I have seen your webpage. I would like to ask you if you know what
can be reason to have empty output evenif I active -b 1 in both svmtrain
and svmpredict functions? Evenif I created the data randomly, I do not
know why but the output is empty. Do you have any idea?

19. Xu Cui says:

@Demir
Demir,

Unfortunately I am not sure what happened. I tested on my side and it works. Please note the last output variable of svmpredict is probability.

20. tom booth says:

Hi Xu Cui and other users,

This is great. Thanks for sharing it. In the last few months has anyone developed any code for 2 class plots bigger than 2D?

Would be great.

Also has anyone worked out code for accuracy confidence intervals (such as by bootstrapping?)

Thanks,

Tom

21. amira says:

i have 6 (labels: 1 2 3 4 5 6) classes and the data dimension is 262 i use libsvm to classify the data labels
first is this labeling is correct ?
how i can the classes result from svmpredict model

22. Zaidi says:

Dear Xu Cui
I want to use LIBSVM through MATLAB. I have a MS Visual C++ 2008 compiler.But the LIBSVM package is not opening in MATLAB. Please help.

23. I get an error saying;
”??? Error using ==> svmtrain at 172
Group must be a vector.”
in the cv = svmtrain(l, d, cmd); line,
what’s wrong? i’m trying to understand this svm code.
Regards,

24. Betty says:

Hi Xu,

I’m pretty new in Libsvm (Matlab interface) and SVM in general.I was able to write a little code but I’m not sure if it works correctly and I have some other questions, too. Is there a possibility to send an e-mail to you?

25. Xu Cui says:

@amira
amira, please refer to the 3 class example

26. Betty says:

27. jack says:

Hi,

I have a question regarding non linear SVM. does the LibSVM package provide the Support Vectors in the higher dimensional space or not? I need to know what is the coordinates of samples when transferred to higher dimension space by SVM.

Thank you

28. Nicola says:

If I have only one data set how can I determine how much my model is good? Is “non sense” try to predict the model I’ve built on the same dataset that has been use for the training?

29. Suresh Gorakala says:

Hi Xu,

How to determine the values of the for loops :
for log2c = -1:3,
for log2g = -4:1,

Thank you

30. Xu Cui says:

@Suresh Gorakala
Usually you need a few steps. First step you will try big ranges, e.g. -5:5, then after you find peak, narrow down the range with higher resolution around the peak, and repeat.

31. SureshGorakala says:

@Xu Cui
Could you please eloborate the above explanation.

32. samir says:

Hi
I want to use libsvm with iris dataset witch have 4 features and 3 classes, I want to do 1 vs 1 and 1 vs all. I find W and b for each 2 class in 1 vs all but I cant plot the result, please help me ?!

33. Ali says:

HELP.

34. Ali says:

Dear Xu
I want to built an SVR for solar cell modeling where I want to predict the “voltage” in terms of “current” and “temperature”, namely, V=f(I,T). I have worked with neural networks and I know the meaning of training and testing. I used your provided code and faced with some problems. For example, suppose that x_train=[1 2;3 4;5 6] and y_train=[5;6;8] (target). how can I built a SVR model to predict the voltage for the point of x_test=[1.4 3]. please give me the matlab codes.
I will be very thankful in advance.
Ali

35. Marcus Fwu says:

Hi Xu

I have some questions regarding svm and I need your help and expertise.

1) Based on the first image (linear shown above) which is obtained via training/testing data using libsvm, May I know what does the axes of the graph represent?

2) Is it alright to use a linear kernel for a two-class classfication that has a total of 69 features/dimensions?

As I am still new to svm, I am not sure if i am asking the right question. I apologise if I am not accurate or specific enough. I sincerely thank you for reading this comment and your help will be deeply appreciated.

Marcus

36. Xu Cui says:

@Marcus Fwu
1) they are the two dimensions of the data.
2) It depends on the nature of your data. Usually when the # of features is big, non-linear kernel may overfit.

37. Jonathan says:

Hello Xu Cui, I need to make a classification of satellite images, I’m use LIBSVM in MATLAB, but I have 2 question:

1. How can i use a classification “1 vs 1” and “1 vs rest”? ¿what commands can i do use?
2. In my case I have got 3 classes, How can i use cross validation ?

38. Hello Sir,
I have developed svm_model for regression, now i want to test it for a given input’x’. after reading the readme file i got that i should use
Function: double svm_predict(const struct svm_model *model,
const struct svm_node *x);
but i do not know how to use? please help so that i can use it in matlab.
thanks and regards
Vishal mishra

39. sapna says:

hello sir,
i am using libsvm for the classification of human actions. i have selected to actions and want to classify those actions so it is a binary classification.
i am facing the problem that libsvm shows 100% accuracy but the decision values are constant and hyperplane is not shown on graph
it gives me the warning.
Warning: Contour not rendered for constant ZData

Thanks

40. sandhya says:

How to scale both training and testing data in same manner?

41. ahmed says:

hello
I would like to ask you what are the best svmtrain parameters for phoneme recognition

42. shah says:

Hello XU Cui!

Can you please guide me how to create test and train files for libsvm use I have image data set, do not know how to divide it into test and train sets?

43. Sami says:

Hello Xu Cui,

I really dont have a question but I just wanna say thank you. you made a great explanation and very helpful. I couldnt find any perfect examples as yours

44. Sami says:

Hello Xu Cui,

I have two classes that have 300×200, 300 instances and 200 is the dimensionality of the feature. I hope you help me figure out how to represent this in a two linear class(decision boundary and # support vectors in each class.

Thank u

45. Nanpui says:

Dear Xu Cui,
I am very confused about one point. Until now I was always using test_label in SVMpredict command,where you have used zeros. Is it wrong what I am doing? I always get 0% accuracy, when used zeros or any double values.
I checked read_me file it also says if you dont have test labels you can use any double value. But when I use different values I get 0 % accuracy.
Thanks a lot.
Regards,

46. Nanpui says:

Thank you so much for this priceless information. I was wondering why do I get zero accuracy when I used the half of the training data in testing with zero labels as new data? I would be more than happy if you could kindly reply to my massage. Thank you

47. Nanpui says:

I just realized, I have asked the same question twice, I am really wondering why I get the same problem and is there a particular reason why it is happening:( thank you in advance.

Dear Xu Cui,
Can you please guide for inputting satellite images into matlab for libsvm usage

49. krupal says:

In my classification problem, I have 6 classes. When I use libsvm and use predict function, it will give 15 decision values for each input. From these values how can I decide predict label class?
Also, is these values representing the values of decision funtion?

50. Jayshree Kundargi says:

Hello Xu Cui,
I have learned cross-validation to learn optimal values for c and gamma for radial basis function SVM using LIBSVM and the code available at LIBSVM site. But if i want to use the same code to obtain value of only C for a linear SVM it does not work.Some matrix dimension mismatch is the error message. Can you help in this regards. Tnx.

51. Afsheen says:

@serra
MATLAB has built in ‘svmtrain’ and ‘svmclassify’ functions… but here in this code Author has used the ‘svmtrain’ and ‘svmpredict’ functions from libsvm package available at https://www.csie.ntu.edu.tw/~cjlin/libsvm/

52. Debesh Jha says:

How to determine the values of the for loops :
for log2c = -1:3,
for log2g = -4:1,

How to know the optimum range for the value of c and g?
when i tried with different range i get different accuracy and different cross validation accuracy.(low cross validation like 52.4 )but accuracy 90 and sometime even 100% . how to know whether i have overfitting in my data?

53. Hello