The human object segmentation and classification are main work in the applications of Intelligent Visual Surveillance System or Passenger Flow Counting System. Traditional approaches to segment and classify human objects are usually based on the face, leg motion and silhouette. These algorithms' performances and their applications have proved to be effective in recent years. But these algorithms all assume that features can always be extracted. In complex situations, however, features adopted in traditional algorithms might not be extracted, because human attitude and illumination change greatly. In this case, if a definite feature is used, the algorithm's accuracy will fall. In this paper we propose an approach to select the feature and the corresponding algorithm adaptively based on the human attitude and object neighborhood illumination. The selected features can be used in the following tracking operation. Because this method solves the human object segmentation and classification problem, it can broad the 3D recovery and behavior understanding research results in simple situations to the application in complex situations. In this paper, the algorithms are proposed for the human attitude and illumination detection, the feature selection strategies in different situation are given. The experimental results show that the algorithm can detect the object lightness properly, and can give the right attitude for feature selection. The algorithms have good performance and computation efficiency.© (2008) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.