LONDON—The Bank of England waded into the highly charged debate on the nation’s membership in the European Union, saying the possibility that Britain chooses to leave the bloc represents the biggest domestic risk to the stability of the U.K. financial system.

In wide-ranging testimony to lawmakers on the U.K.’s economic relationship with the EU, BOE Gov. Mark Carney said Tuesday that uncertainty surrounding the outcome of the coming referendum on Britain’s future in Europe is being felt in financial markets already and that a vote in favor of leaving could cause a short-term hit to the wider economy.

A possible departure represents “the biggest domestic risk to financial stability,” Mr. Carney said, with potential consequences for Britain’s balance of payments with the rest of the world, its housing market, foreign investment and its banks.

Like many other central bankers, BOE officials take pains to avoid being dragged into the political fray.

Officials carefully have sought to frame their comments on the referendum as a dry assessment of how EU membership affects their ability to meet the central bank’s twin goals of maintaining stable inflation and safeguarding financial stability. However, that hasn’t stopped advocates of an EU exit from criticizing the central bank for overstepping the mark.

There were at-times heated exchanges during the almost-three-hour long testimony, with some so-called euroskeptic lawmakers on the panel accusing the central banker of inappropriately venturing into political waters.

Jacob Rees-Mogg, a lawmaker in Prime Minister David Cameron’s Conservative Party who favors quitting the EU, accused Mr. Carney of “political partisanship” that threatens the central bank’s “Olympian detachment.”

Mr. Cameron is campaigning for Britain to remain in the EU ahead of a June 23 referendum on the U.K.’s continued membership.

Mr. Carney said he wasn’t giving a view on how Britons should vote and denied the accusations that the BOE has been persuaded to support the pro-EU cause by the government.

“We are expressing views that are the view of the institution. We are not being leaned on by anybody,” Mr. Carney said.

The governor’s remarks, which came alongside testimony from Jon Cunliffe, the BOE’s deputy governor for financial stability and a former senior British official at the EU, represent the central bank’s most public intervention in Britain’s EU debate to date.

Mr. Carney’s clash with euroskeptics on the panel also underscores the fractious nature of that debate. Proponents of quitting the EU say the U.K. would be freer to prosper outside the bloc, while campaigners for Britain to remain a member say the economy would suffer if it lost access to the EU’s vast single market for goods and services.

Mr. Carney revealed that some banks are drawing up contingency plans to shift jobs and activities elsewhere in the EU in the event of a British exit, although he declined to tell lawmakers which institutions he meant, citing “supervisory confidentiality.” He said some financial-sector activity would “without question” move elsewhere in the EU if Britain lost access to the EU’s single market.

Still, the BOE governor said so-called “passporting” arrangements that would allow continued access to the single market in the event Britain left the EU should be possible to achieve.

Advocates of the U.K. quitting the EU say London’s position as one of the world’s pre-eminent financial centers would endure outside of the EU and that leaving may make it easier to broker deals with emerging markets that would benefit financial services.

Mr. Cameron in February negotiated new terms for Britain’s EU membership, which included provisions safeguarding the rights of non-eurozone countries to police their own financial systems, a key British goal amid deepening economic integration among the 19 countries that use the single currency.

While proponents of Britain’s exit from the EU have question how substantive the reforms are, Mr. Cunliffe said these new terms were “material and significant” and should help the bank fulfill its objectives of maintaining low and stable inflation and protecting the financial system. “From the bank’s perspective, the agreement is very important,” he said.

The two officials also detailed the risks the BOE sees surrounding the vote and the potential short-term consequences of a decision to leave.

They said a recent slide in the pound and an increase in investors purchasing insurance against a further decline in sterling are reflections of market nerves about the vote.

Mr. Carney said “a host of asset prices” could be affected by an exit vote and that banks and companies may find it harder or more expensive to borrow money. The BOE on Monday said it has scheduled three extra cash auctions for financial firms in the days surrounding the referendum to see off any concerns around financial-system funding.