Equations set of a class of optimal control problems are deduced in quantum mechanical systems based on variational approach,. Furthermore, an on-off variable metric numerical iterative arithmetic with optimized search step length is used to solve such an optimal problem. Finally, some simulation experiments on a spin-1/2 particle system are done. Experimental results demonstrate the validity of the method proposed. At the same time, the effectiveness of system control under different parameters in the performance index are analyzed.