The chairman of the Joint Chiefs of Staff apologized for participating in Donald Trump’s controversial appearance for a photo at a historic church last week, when law enforcement forcibly cleared protesters.

“I should not have been there,” Gen. Mark Milley said in a prerecorded graduation address to the National Defense University on Thursday. “My presence in that moment and in that environment created a perception of the military involved in domestic politics.”

Milley’s presence has been widely criticized for giving credence to Trump’s claim that he could use the U.S. military to quell domestic protests. He is the latest senior current or former military official to rebuke or distance themselves from the event, during which Trump briefly held up a Bible outside a damaged church before walking back to the White House nearby. Defense Secretary Mark Esper, who was also there, has said he wasn’t aware Trump was setting up for a photo.