Opened in 1926, the original restaurant at 3427 Wilshire Boulevard remains the most famous due to its distinctive shape. Whimsical architecture was popular at the time, and the restaurant was designed to catch the eye of passing motorists. It is said that the shape of the hat worn by New York governor and 1928 Democratic Party presidential candidate Al Smith.
The small cafe, close to popular Hollywood hot spots such as Cocoanut Grove at the Ambassador Hotel, became successful enough to warrant the building of a second branch.
In 1937, the building was moved to 3377 Wilshire Boulevard at the northeast corner of Wilshire Boulevard and Alexandria Avenue. After being sold in 1975 and renovated, it was replaced in 1980 by a shopping center known as the Brown Derby Plaza. The domed structure was incorporated into the third floor of the building and accommodates a cafe.
A Korean mini-mall occupies the site today.