Holding Russian and Ukrainian flags, they shouted: “The occupation of Crimea is Russia’s disgrace.” A smaller pro-Moscow rally was being held elsewhere.
Moscow supports the vote, which Ukraine and the West have dismissed as illegal.
Kiev says Russian “provocateurs” are behind clashes in eastern Ukraine.
Three people have died in pro- and anti-Moscow rallies in the cities of Donetsk and Kharkiv with sides blaming each other.
Russia’s military intervention in the Crimean peninsula – part of Russia until 1954 and host to its Black Sea fleet – followed the fall of Ukraine’s pro-Moscow President Viktor Yanukovych on 22 February.
Mr Yanukovych’s decision in November not to sign a deal on closer ties with the European Union – opting instead for closer ties with Russia – had sparked months of protests.
Both the US and EU have threatened sanctions against Moscow.
On Saturday, the UN Security Council failed to adopt a US-drafted resolution that defines Crimea’s referendum as illegal.
Thirteen members backed the resolution and China abstained, but Russia vetoed its passage.
After two people were killed in Kharkiv on Friday, interim Ukrainian Prime Minister Arseniy Yatsenyuk said on his Facebook page: “Hired provocateurs from a neighbouring state are already playing on this situation and preparing provocations professionally.
“Former Ukrainian leaders, charged with crimes, are funding the planned disturbances in the country’s south-east in co-ordination with the Russian extremist forces. Don’t let them manipulate you.”
Interim President Oleksander Turchinov echoed the charge, saying “Kremlin agents” were organising and funding the protests in eastern Ukraine.Moscow denies this, vowing to protect its “compatriots” from far-right radicals.
Russia’s foreign ministry said in a statement on Saturday that it was “receiving many requests to protect peaceful citizens” in Ukraine.
The ministry said it would consider the requests.