President Robert Mugabe, 89, has said he will step down after 33 years in power if he and his Zanu-PF party lose.
Prime Minister Morgan Tsvangirai's Movement for Democratic Change (MDC) have accused Zanu-PF of doctoring the electoral roll, a charge it has denied.
Campaigning was mostly peaceful, with few reports of intimidation.
Zanu-PF and the MDC have shared an uneasy coalition government since 2009 under a deal brokered to end the deadly violence that erupted after a disputed presidential poll the previous year.
Mr Tsvangirai won the most votes in the first round, but pulled out of the run-off with Mr Mugabe because of attacks on his supporters.
'Determined to vote'
The government has barred Western observers from monitoring Wednesday's elections, but the African Union (AU) and the Southern African Development Community (Sadc), as well as local organisations, have been accredited.
Polls opened at 07:00 local time (05:00 GMT) and are due to close at 17:00 GMT.
The turnout is expected to be high among the 6.4 million people registered to vote, with tens of thousands of people attending rallies in recent weeks. Results are expected within five days.
Wednesday has been declared a national holiday to ensure people can vote. Despite this, voters queued for several hours outside polling stations across the country before they opened, reports the BBC's Nomsa Maseko in Harare.
"I got up at four but still couldn't get the first position in the line," Clifford Chasakara, a voter in the western province of Manicaland, told the Reuters news agency.
"My fingers are numb, but I'm sure I can mark the ballot all the same. I'm determined to vote and have my vote counted."
At a news conference at State House on Tuesday, Mr Mugabe was asked if he and Zanu-PF would accept defeat.
"If you go into a process and join a competition where there are only two outcomes, win or lose, you can't be both. You either win or lose. If you lose, you must surrender," he said.
But Mr Tsvangirai dismissed the president's remarks.
"He does not believe in the right of the people to choose. He does not believe he can be voted out of office," he told the BBC.
The 61 year old has vowed to push Mr Mugabe into retirement; it is his third attempt to unseat him.
An MDC spokesman said separately that the party was only prepared to accept the results of the elections if they were "free and fair".
On Tuesday, the MDC accused Zanu-PF of doctoring the roll of registered voters, which was released by the Zimbabwe Electoral Commission (ZEC) only on the eve of the polls after weeks of delay.
The MDC claimed the roll dated back to 1985 and was full of anomalies
A BBC correspondent has seen the document and says it features the names of thousands of dead people. He says many names with the same address appear two or three times.
A Zanu-PF spokesman denied the allegations and pointed out that appointees from both parties were on the ZEC. He also accused Finance Minister Tendai Biti, a member of the MDC, of not funding the commission properly. The ZEC has not commented.
In addition to Mr Mugabe and Mr Tsvangirai, there are three other candidates standing for the presidency - Welshman Ncube, leader of the breakaway MDC-Mutambara; Dumiso Dabengwa of the Zimbabwe African People's Union (Zapu), and Kisinoti Munodei Mukwazhe, who represents the small Zimbabwe Development Party (ZDP).
To be declared a winner, a presidential candidate must win more than 50% of the vote. If no candidate reaches this mark, a run-off will be held on 11 September.
The elections will be the first to be held under the new constitution approved in a referendum in March this year.