Burglars broke into De Beers and Omega at the Royal Exchange in the City last night.
City of London police said they did not yet know the value of the stolen jewellery.
A spokesman said: ''Just after 10pm last night De Beers and Omega were broken into at the Royal Exchange and a burglary took place.
''Police are now investigating the incident.''
The Royal Exchange, built in 1565, is located between Threadneedle Street and Cornhill and boasts more than 30 luxury stores. A security guard working at the imposing white stone building said staff had been told the robbers entered through the back of the property.
The man, who did not want to be named, said: "We've been told that at about 10pm last night a BMW turned up at the back entrance, four men jumped out and managed to get inside the building somehow.
"At the moment we don't know how much has been taken, but with the price of some of the stuff inside these shops just a few items could easily run up to a very, very large amount.
"It's looking as though whoever has done this has been targeting the shops for a long time."
City of London police refused to be drawn on whether they are investigating claims that the robbers had arrived in a BMW.
A police spokesman said it was still too early in the investigation to determine the cost of the items taken.
A number of Royal Exchange security staff were the only people inside the building this morning.
The three men could be seen taking photographs and making measurements.
The up-market shopping centre, which is closed at the weekend, lies just yards from the Bank of England and sits in the heart of the City.
The front of the Grade 1 listed building boasts a vast Corinthian style eight pillared portico, with the entrance protected by sprawling green wrought iron gates.
The shopping centre, which houses high-end stores such as Gucci, Hermes and Cartier, specialist chocolatiers and tobacconists, has three other smaller entrances, each of these protected by iron gates.
The robbery comes almost a year after raiders stole £40 million worth of jewellery from Graff Diamonds in London's Mayfair district.