The shadows similar to the vehicle and the spots caused by vehicle lamps need to be accurately detected in the vehicle segmentation involved in the video-based traffic parameter measurement. Generally, the road surface is different from the vehicle surface in the gray-level architecture. An invariant gray-level architecture-the extremum image in the changing illumination environment is derived and a novel algorithm is presented for detecting shadows and spots. The gray-level structure that is not sensitive to the illumination is employed in the algorithm and the road surface mistaken as vehicles can be removed.