The Morse’s Teas building at the apex of Hollis and Lower Water Streets was not the first building at that location. In 1753, three years after Halifax was founded, British army agent Thomas Saul had a large stone house built on the property. The house's next owner converted it to a coffee house called the Jerusalem Coffee Shop, which was a popular spot for merchants. In 1837, a fire engulfed the street and destroyed the building along with many others. In 1841, Edward Starr and Co. bought the property and erected a four-story ironstone warehouse using the surviving bricks from the original house. The building was constructed in the Georgian style with a flat roof, pronounced corner quoins, and simple granite trim. It was known as the Jerusalem warehouse after the former coffee shop and used by several Halifax businesses. In 1855, the warehouse was purchased by J. S. McLean, who owned a grocery store that specialized in the sale of imported tea, which he distributed throughout the Maritimes. After McLean, the building changed hands several times before being purchased in 1910 by O. E. Smith, President of J. E. Morse & Company Ltd.
In 1879, John E. Morse (1840-1933) opened a small tea import business on Halifax’s waterfront. He saw tea drinking as an important part of English society and wanted to bring quality teas to Canada. Morse sourced most of his tea blends from India and China. Around 1890, O. E. Smith, a commercial traveler, joined the company. Within ten years, Smith was in charge and Morse had moved to Wolfville.
The company grew under Smith's direction, and their new warehouse allowed them to import and ship even larger quantities of teas. Smith made trips around the world where he stopped at tea growing countries to find the best tea blends for the company. J. E. Morse & Co. focused on customer satisfaction, providing buyers with a booklet called "Tea Tips," which educated them on the history and origins of tea and how best to prepare it. Employees were given instruction manuals on how to operate the machines that packaged and weighed the tea. For fear of dampening the quality of the tea, they were asked not to operate the machinery unless they understood how it worked. Due to the expensive cost of freight, J. E. Morse & Co. only shipped tea in wholesale quantities. Smith made several changes to the Morse's Teas building over the years, most notably following a fire in 1927, when he added two additional stories making it six bays tall as well as wide.
While J. E. Morse & Co. remained in Smith's family for several decades, in the late 1960s, their business was threatened by plans for Halifax's "urban renewal." The City of Halifax wanted to build a highway and interchange along the waterfront, and the Morse's Teas building was directly in its path. Many buildings nearby had already fallen to the wrecking ball and others were at risk. While the warehouse was eventually saved, the ordeal put strain on the company.
Around 1982, J. E. Morse Co. was sold to G. E. Barbour Company Ltd. of Sussex New Brunswick. Four years later, they vacated the Hollis Street warehouse, which had recently been declared a heritage property. The building sat empty until the late 1980s when it was purchased by the Nova Scotia College of Art and Design and restored. In 2004, the Morse's Teas building was sold to a developer who painted over the iconic Morse's Teas sign – the last vestige of the company's home in Halifax.