"I am responsible for the defeat and has taken the decision to quit as the national coach with immediate effect," Dollah, 51, told reporters at KLIA after returning with the team late Thursday.
With the thrashing in the qualifying second round Asian Zone Group A match at the Mohamed Zayed Stadium in Abu Dhabi, Malaysia hit a new all-time low in their football history.
It was the worst result for Harimau Malaya since going down to New Zealand 8-2 in an international match played in Kuala Lumpur almost 50 years ago.
The team will meet Saudi Arabia on Tuesday in the qualifications.