Motion estimation with a quad-tree variable block size is the driver for the high performance of HEVC in video compression. However, tremendous sum of absolute difference (SAD) computations are needed for every single block size, resulting in huge memory access and power consumption. This work proposes an early termination mechanism at the hardware level that is suitable for quad-tree variable block size motion estimation without degrading video quality. The simulations show reductions in memory access to fetch pixels of about 34.03% and 51.98% saving of SAD computations. This reduction results in reduced power consumption. The simplicity of the algorithm makes it a hardware friendly mechanism, which could be adopted in any motion estimation hardware design.