Mattis made the comments as he began a one-day visit for talks with Israeli leaders, who strongly supported the recent US strike against an airbase in neighbouring Syria over an alleged chemical attack on a rebel-held town.
"The bottom line is there can be no doubt in the international community's mind that Syria has retained chemical weapons in violation of its agreement and its statement that it had removed them all," Mattis said during a press conference with Israeli Defence Minister Avigdor Lieberman.
"It's a violation of the United Nations Security Council resolutions, and it's going to have to be taken up diplomatically, and they'd be ill-advised to try to use any again. We've made that very clear with our strike."
Mattis added that Syria had "dispersed their aircraft in recent days."
An Israeli military assessment has found that Assad's regime was still in possession of "a few tonnes" of chemical weapons, an army official confirmed.
Some Israeli media reports put the number at between one and three tonnes. Lieberman declined to comment on the assessment at Friday's press conference.
Assad, backed by his ally Russia, has strongly denied the allegation that his forces used chemical weapons against the rebel-held town of Khan Sheikhun on April 4, describing it as a "100 per cent fabrication".
He has said repeatedly that his forces turned over all chemical weapons stockpiles in 2013, under a deal brokered by Russia to avoid threatened US military action.
The agreement was later enshrined in a UN Security Council resolution.
Talks with Netanyahu
Mattis was later holding talks with Prime Minister Benjamin Netanyahu in Jerusalem, followed by President Reuven Rivlin.
Israel and the United States have long had close strategic ties, with Washington providing Israel more than $3bn per year in defence aid and President Donald Trump pledging unstinting support for the country.
Despite tensions over Israeli settlement building, Barack Obama's administration signed a new agreement with Israel before he left office increasing the amount to $3.8bn for a 10-year period beginning in 2018.
Mattis hopes to hear directly from Israeli leaders on their concerns and what they expect from the Trump administration, a US defence official said.
In a further sign of close relations, Israel is to receive three more F-35 stealth fighter jets on Sunday, adding to two which arrived in December.
They are among 50 that Israel has agreed to buy from US aerospace giant Lockheed Martin.