His comments came after the Israeli-Palestinian clashes in and around the holy site over the weekend that left 170 people injured, mostly Palestinian protesters at the hands of Israeli security forces.
The violence comes nearly a year after similar tensions sparked an 11-day conflict between Israel and militant groups in the Palestinian enclave of Gaza.
Al-Aqsa is the holiest site in Judaism and the third holiest in Islam.
The weekend of violence has further heightened tensions, with Israel launching its first airstrike on the Gaza Strip early Tuesday in response to a rocket fired from the Palestinian enclave.
Erdogan said on his official Twitter account that he told Herzog “the fact that the Al-Aqsa mosque was raided by fanatic groups yesterday and the day before after morning prayers…
He said that “These images, seen every year because of some radicals, hurt the conscience and provoke justified reactions throughout the Islamic world.”
Erdogan told the Israeli president: “In this sensitive period, I would like to reiterate the need not to allow provocations and threats against the status and spirituality of the Al-Aqsa Mosque.”
The Turkish leader reiterated his call on everyone to “do the utmost” to preserve the spirituality of the holy site.
Israel and Turkey have proclaimed a new era in relations after more than a decade of diplomatic rifts after Herzog paid a historic visit to Ankara in March.
Erdogan, an outspoken supporter of the Palestinian cause, had criticized Israel’s policy toward the Palestinians in the past.
Turkey maintains ties with Hamas, which has controlled Gaza since 2007.