I am solving a very huge matrix and I would like to know how to increase the solver speed.
Wouldn't we all !!
I think a bit more information is required about what you call a "solver".
For my take on a solver, I have tried to implement vector instructions and use OpenMP to improve run speed.
The smart money is probably on using a library solution from MKL, IMSL or something equivalent, as they have solved using modern processors in a much better way than I have achieved, especially when managing cache effectively.
Then again it might be the hardware approach to buy more memory and a faster disk.
You really need to describe you problem and the solver in a more detailed way, as the brevity of your question suggests you have a lot to understand about the question you are asking.
Yes, you need to describe what you mean by "solving a matrix." Are you solving a system of linear equations? I sincerely hope you are not trying to do this by straight matrix inversion, for there are much faster ways of doing it, and different ones to take advantage of certain characteristics of the system, e.g. Is it symmetrical? Is it banded? Is it sparse? And so on. These methods are widely used and well documented, and you can find them in MKL or ISML or elsewhere, as already mentioned.