In this Jan. 6, 2021, image from video, Alan William Byerly, right, is seen allegedly attacking an Associated Press photographer during a riot at the U.S. Capitol in Washington. Byerly has been arrested on charges that he assaulted an Associated Press photographer and police officers.

WASHINGTON — The fencing installed around the Capitol after the deadly Jan. 6 insurrection will start being removed as soon as Friday, but most visitors are still not allowed inside the iconic building.

That's according to the House sergeant-at-arms in a memo Wednesday to all members of Congress and staff. The Associated Press first reported last week that the fence was coming down.

Even with its removal, the Capitol will remain closed to most visitors, the memo said.

“Although the temporary fencing will be removed, current building access restrictions will remain in place,” according to the memo.

Five people died in the January attack, including a Trump supporter shot and killed by police inside the Capitol and three people who suffered medical emergencies in the crowd. One police officer died later after having battled rioters. Two other officers later took their own lives.

The U.S. Capitol Police will continue to monitor intelligence information and potential threats, and new fencing could be swiftly erected, the memo said.

“The Architect of the Capitol has the ability to and will expeditiously reinstall the temporary fencing should conditions warrant,” according to the memo.

The Capitol has been closed to most visitors for the longest stretch in the nation's history, the one-two hit of the coronavirus outbreak that shuttered operations last spring and the insurrection that kept it off limits. It's now nearing 16 months.