Malian army spokesman Diaran Kone told the BBC that between 20 and 40 militants had entered the city on Wednesday night between 23:00 and midnight.
After hours of explosions and gunfire, French and Malian forces then opened fire at the city hall with heavy machineguns, Reuters new agency reported.
Gao's mayor Sadou Harouna Diallo told the BBC he was hiding inside the city hall and could hear shooting outside at about midday.
A spokesman for Mujao, the Islamist group which controlled Gao for nine months, said their fighters had entered Gao on Wednesday.
Ghana News Headlines
For latest news in Ghana, visit Graphic Online news headlines page Ghana news page
"Our troops have been ordered to attack. If the enemy is stronger, we'll pull back only to return stronger, until we liberate Gao," Abu Walid Sahraoui told AFP.
He later said that Mujao had detonated a car bomb near a military base in Kidal.
He told AFP the group had no difficulty getting into Kidal "to blow up a vehicle as planned".
"More explosions will happen across our territory," he added.
The BBC's Maud Jullien in the capital, Bamako, said early reports suggested a black car drove up to a private residence about 500m (1,650ft) from a camp where French and Chadian soldiers are based and exploded, killing one person and injuring two others.
Earlier this month, two suicide attacks in Gao - the first in Mali's history - followed by fierce fighting in the city have led to fears it may be a long battle to bring security to the vast desert region.
France has said it wants to start withdrawing its 4,000 soldiers from its former colony next month - and would like the African-led mission to become a UN peacekeeping operation.