The mosque was built by and named after an Arab merchant from Hadhramaut (a valley on eastern Yemen) Syed Omar bin Ali Aljunied (1792 – 1852). After Sir Stamford Raffles founded Singapore in 1819, he invited Syed Omar and his uncle Syed Mohamad, who was a successful businessman in Palembang, Sumatra to set up their business in Singapore. After arriving in Singapore, the Aljunieds became major landowners and philanthropists and shared much of their wealth with the community at large.
The original mosque was established in 1820 at the southern end of Kampong Melaka. The area then became a social hub for the Malay community as well as Muslims originating from India, the Middle East and Southeast Asia.