Twin explosions and shots could be heard just moments prior to the gunmen attempting to enter the site of the Indian consulate, the spokesperson for the local governor said.
While a consulate official told the AFP news agency by telephone from inside the compound that "we are being attacked. Fighting is going on."
There were no immediate details of casualties or damage.
Sunday's attack comes as Indian security forces try to contain an attack on an airbase near the border with Pakistan which has seen more than seven military personnel killed and 20 others wounded.
Mazar-i-Sharif is located some 430 kilometers (270 miles) northwest of Afghanistan's capital Kabul.

jlw/rc (Reuters, AFP)