inot Hall's origins date back to 1858. In October of that year, Stephen Smith purchased the lot currently known as the Smith Block. It is assumed that the building was built in 1859. By 1860, Minot Hall was listed in the Boston Public Directory. An 1866 mortgage note on the property described it as a function hall with four stores on the first floor. By 1899 photographs of the building show the Smith Block as the Hotel Olympia, with signs in the upper windows saying "Hall Lodge Room to Let". Records indicate that the upper floors and ballroom were also used as a theatre, social club, and a billiards hall. Over the years, the businesses that have occupied the first floor storefronts include a newsstand, barbershop, Chinese restaurant, diner, pawn shop, and jewlery store.

The building originally had four storefronts, with each of the bays having elaborate 14 foot high tin ceilings and tin wallpaper. The floors were covered in turquoise and white tiles, and there were hand painted scenic tiles on the walls. A grand marble staircase led to the second floor, which had a large common room, which was a "smoking room" and several smaller rooms. This room had elaborate 12 foot high tin ceilings and tin wallpaper. In the center rear of the smoking room was a large winding wooden staircase that led to the third floor ballroom. The ballroom had 22 foot high ceilings with elaborate gilded mirrors and maple floors. One end of the room had a balcony, and the ceiling had a huge plaster rosette that housed an elaborate chandelier. A smaller stairway led to the fourth floor balcony, and then to the fifth floor. On the fifth floor was a kitchen area with soapstone sink, a wainscoat ice box, and a dumbwaiter. There was stenciling around the borders of the walls and gas light fixtures.

In 1982, several fires broke out inside of the building, destroying the interior. An attempt was made to subdivide the building into 10 loft spaces, but the project did not succeed. The building was finally purchased in 1997 and renovated into the Minot Hall Antiques Center. The building is currently being renovated, with a large wrap-around addition, as The Residences at Minot Hall.