When Charles l'Enfant designed the Fort Mifflin that was built in the 1790's, he included a Citadel. Most American seacoast defenses at the time were required to include a citadel, built atop an ammunition magazine. There's no evidence that such a magazine ever existed below this citadel, but it was initially outfitted with two-foot-thick walls with 45 gun portals and a brick parapet along the roof, to serve as a protected lookout position. The Citadel also had a long flagpole, which was to function as a means to communicate with the city of Philadelphia should the need arise. The original Citadel was completed in 1796.

And then the Citadel concept was scrapped during the fort's renovations of the 1830's, because the fort's commanding officer needed a cool place to live! By 1815 this building was being referred to as the Commandant's House, and by 1835 any idea of using the structure as a last-ditch stronghold was forgotten. 'Twas refurbished for comfort and grandiosity.