Shadow detection is a well-known problem in image and video processing. Detecting moving shadows is useful in numerous applications such as object detection and tracking. Most works in this area are not suitable for shadow detection using a low-cost outdoor surveillance camera. In this work we suggest a fast shadow detection approach for video surveillance, by comparing each video frame to a continuously updated background image. We differentiate shadow areas from foreground areas by assuming the shadow pixels are associated with background pixels through a nonlinear tone mapping. This assumption is general and applies to various systems and scene conditions. A distance measure between patch images that account for nonlinear tone mapping is calculated by adapting a recently suggested approach for pattern matching termed Matching by Tone Mapping (MTM). We show that the proposed technique is computationally efficient and outperforms state-of-the-art shadow detection techniques in typical surveillance scenarios.